We have been developing software in Julia for over 5 years now, and had been writing C and C++ code for over 15 years before that. Julia is a superb language and package ecosystem for science, engineering, research, and even software development in general. The language also very elegantly empowers users to take advantage of compile-time optimization capabilities without requiring an understanding of complicated (and nearly unreadable) template meta-programming techniques like C++ does (for those not familiar with that, just take a quick glance at the source code for CGAL or Eigen).

While Julia has solved some rather challenging problems, it is still a relatively young language. Many organizations already have trusted solutions written in C or C++, and while Julia presents a powerful new capability, some of the features that they rely on are missing or not yet mature enough. Often users wish to use their existing C-based software from within Julia and migrate their implementations to Julia as time or funding permits.

Julia’s built-in facilities for binding with C libraries are thoroughly explained in Calling C and Fortran Code in the Julia documentation. Please start there if you are not yet familiar with them since this article assumes you have at least a basic working knowledge of interfacing C with Julia.

Even though Julia provides facilities for using C libraries, just about every practical C API exceeds those facilities, and the Julia documentation notifies developers of this fact. Unfortunately, this is one place where some of the new users encounter a bit of tool friction when adopting Julia. Having worked through these challenges, we thought it would be beneficial to highlight the following pain points so that you can avoid them.

Wow! That seems like a long list of rather common C constructs which are not represented in Julia, doesn’t it? But there is some good news! We have developed a package, CBinding.jl, to add proper support for C (ANSI C, C89, C99, C11, and C18) constructs to Julia. Let’s have a look at each pain point in detail before we present how our package brings proper support for C to Julia.

TLDR-ers can just go here…​

Many C libraries use union, so it is frustrating to integrators when there is no support for them in Julia. A union specifies the subset of types that a field may have, and it can be used as an idiom to mimic polymorphism, which the following code illustrates.

In order to use Number, Julia code must drop the type information of the nested union and replace it with a Cdouble (the largest of its fields). The code using the struct must use reinterpret calls along with other byte-level intrinsic operations to try to get the behavior they desire. This is not a very elegant solution for users to encounter.

Enumerations are as ubiquitous as unions in C API’s. Julia provides the @enum macro to define Julia’s enumerations which are, unfortunately, not compatible with C’s enumerations. A user is directed by the documentation to use integer primitive types and a bunch of constant definitions to mimic the behavior of C. The CEnum.jl package claims to provide a "C-compatible" enumerations for Julia, so we used it initially to interface with any enum defined in a C header file. However, we eventually discovered that it doesn’t entirely comply with C enum semantics, and the following example is one that exposes that.

The package is not able to represent enumerations having multiple labels for the same value, and it doesn’t correctly determine the integer type used for storing an enumeration (which is actually derived from the enumeration’s values). An enum could be of type Cint, Cuint, Clonglong, or Culonglong depending on its set of values, but CEnum.jl leaves that detail to the user to know and work around.

Nesting a struct or union is pretty standard in C API’s. In fact, Julia’s header file offers many examples of composition using both constructs, like this one:

Actually, that small snippet of code demonstrates several C features which have no representation in Julia. It uses bit fields, an anonymous nested union, and composition of a struct within a union. A more simplistic illustration of a nested struct would look like this:

In Julia, a nested struct must be immutable, otherwise the outer struct will store a reference to the nested struct. If the nested struct is also used as a top-level struct in the C API, then it must be declared mutable. To compensate for the contradiction in mutability, both variants of the type must be defined in Julia. The following code shows the approach recommended by the Julia documentation to support the simple C code above.

This simple example begins to reveal how confusing and unmanageable the code can become when porting many nested aggregate types to Julia.

Anonymous fields are also frequently used in types defined by C API’s. They let developers avoid name pollution, particularly when the type is only used once in a very specific context. The first pain point serves as an example using both an unnamed enum type and an unnamed union type. All unnamed types must be given names for their Julia representation.

Using a packed alignment strategy is a feature that rarely affects C development, but it is one that system-level code relies on. The following example of this comes from the ALSA SoC header file:

Usage of __attribute__((packed)) on the struct informs the compiler to tightly pack its fields, and it also enables tightly packed arrays of the struct. It is like telling the compiler to disregard any performance impacts cause by memory misalignment in order to minimize the memory footprint of the struct. There is no capability for changing the alignment behavior of Julia types, just as the documentation says, so interfacing a system-level API, like ALSA, is prohibitively difficult.

In general, integer bit fields are used to reduce a type’s memory footprint, and their usage is common in system-level development where bit-twiddling is a standard modus operandi. Actually, we can look in Julia’s own header file for an example of something the language itself cannot properly represent:

Another property of integer bit fields is that when they are declared using signed integers they use \(N\)-bit two’s complement, where \(N\) is the number of bits specified by the field. This detail is rarely used by C API’s, and few developers know about the behavior. Therefore, it illustrates that when no authoritative solution exists in Julia, developers will implement C features on their own that are likely not compliant with C.

A popular C idiom is to accept user-provided callback functions in order to enable functional composition and customization within the library’s algorithms. Another common idiom is to store function pointers within a struct to create a form of object-oriented interfaces. We can find some good examples of these idioms, shown below, in C frameworks like GLFW, SDL, and Libav.

As directed by the Julia documentation, a user should simply use Ptr{Cvoid} to store function pointers, but that leads to fairly unsafe and fragile code. When using a type-safe, and high-level language like Julia, a user would rather be productive with their development time instead of managing type information or debugging segfaults.

The last pain point that we will highlight is the lack of support for variadic functions. Variadic functions allow users to pass arbitrary numbers and types of arguments to them. A classic example of this comes from the C standard library itself:

Not a lot of C libraries use variadic functions, but some very important libraries (like GLib, GTK, Gstreamer, and others) use them prolifically. When a critical function from such a library cannot be called from Julia, it makes some, or possibly all, of the features of that library inaccessible to Julia users.

So now that you have a good sense of the struggle that a Julia developer faces when interfacing a C library, we wish to introduce you to our work on CBinding.jl. Below, we show our pain-free implementations of all of the pain point examples presented earlier. You will notice that they all have a very natural representation in Julia, yet they still look very similar to their original C form. Additionally, all of these capabilities are fully supported without requiring a Julia developer to do the work of the compiler.

CBinding.jl is the first complete solution for interfacing C API’s from Julia. We believe the package is a critical addition to the Julia ecosystem and it will make any Julia-C integration achievable. We will be publishing future blog posts to demonstrate interfacing real-world C libraries from Julia using our package.

If you are considering the transition to Julia, but have several C libraries you depend on, let us help! Analytech Solutions offers many years of experience working with both Julia and C, and we can streamline your transition process. Please contact us for more information!

Keith Rutkowski is a seasoned visionary, inventor, and computer scientist with a passion to provide companies with innovative research and development, physics-based modeling and simulation, data analysis, and scientific or technical software/computing services. He has over a decade of industry experience in scientific and technical computing, high-performance parallelized computing, and hard real-time computing.