Rust and GoLang are two powerful systems programming languages known for their performance and safety. While Rust offers memory safety and concurrency with zero-cost abstractions, GoLang provides simplicity and ease of use with built-in support for concurrent programming. Combining these two can leverage the best of both worlds. In this blog, we will explore how to call GoLang functions from Rust using Foreign Function Interfaces (FFIs).

What is FFI?

FFI, or Foreign Function Interface, allows programs written in one language to call functions or use services written in another language. This interoperability is crucial for integrating systems written in different languages, optimizing performance-critical sections, or reusing existing libraries.

Why is this Useful to Us?

Our open source PebbleVault Package utilizes an in-memory database called BuntDB, which is implemented as an embeddable database for GoLang projects. In our case we choose BuntDB because of its simplicity, speed and flexible indexing capabilities (Such as spatial indexing).

Our game server however, is written in rust. This presented us with a choice: we could either use a socket connection to forward data between a the rust server, and an independent Go sidecar, Like we do with the Horizon sidecar exe on the client side, or we could spend a bit more time and implement FFIs to get drastically better performance. For us, the choice was clear; FFIs were without a doubt the way to go.

Using Sockets as an Alternative to FFIs

While using Foreign Function Interfaces (FFIs) to call GoLang functions from Rust is a powerful technique, it isn’t always necessary. For many use cases, especially those that are not performance-critical, using sockets as an alternative can be sufficient and even advantageous.

What are Sockets?

Sockets provide a way for programs to communicate with each other, either within the same machine or over a network. This communication is done using the standard networking protocols, typically TCP or UDP. Sockets are a fundamental part of networking and inter-process communication.

Advantages of Using Sockets

1. Language Agnostic: Sockets allow different programs written in different languages to communicate with each other without worrying about language-specific interoperability issues. This makes it easier to integrate systems written in various languages.
2. Decoupling and Flexibility: With sockets, the components of your system are decoupled. This means you can update, replace, or scale them independently without affecting the other parts. For instance, you can rewrite a service in a different language or deploy it on a different server.
3. Simplicity in Deployment: Using sockets for communication can simplify deployment. Each service can be deployed independently as a standalone application, and they can communicate over a network. This is particularly useful in microservices architectures.
4. Ease of Debugging and Testing: Networking tools and techniques are well-established. Tools like tcpdump, Wireshark, and various mock servers can be used to debug and test socket-based communication easily.

When Sockets are Not Sufficient

While sockets offer many benefits, they may not be suitable for all situations, particularly those requiring high performance and low latency. Here’s why:

1. Performance Overhead: Sockets involve network communication, which introduces latency and overhead due to context switching, data serialization/deserialization, and network stack processing. For performance-critical applications, this overhead can be significant.
2. Complexity in Handling Connections: Managing socket connections, especially under high load or with complex communication patterns, can add complexity to your code. Issues like connection management, retry logic, and fault tolerance need to be handled explicitly.
3. Security Considerations: When communicating over sockets, especially over a network, security becomes a concern. Ensuring secure communication through encryption (e.g., TLS) and handling authentication/authorization adds to the complexity.
4. Latency Sensitivity: For applications that require real-time performance, such as high-frequency trading platforms or real-time gaming, the latency introduced by socket communication can be a deal-breaker.

Prerequisites

Before we dive into the technical details, ensure you have the following setup on your system:

1. Rust (with cargo package manager)
2. GoLang
3. Basic understanding of both Rust and GoLang

How we did it

First, let’s write a simple GoLang library. Create a new directory for the project and navigate into it:

Create a file named mylib.go with the following content:

package main

import "C"

//export Add
func Add(a, b int) int {
    return a + b
}

func main() {}

This Go code defines an Add function that we want to call from Rust. The //export Add comment before the function is necessary for the cgo tool to recognize the function as an exportable symbol.

Compile the GoLang Library:

Compile the Go code to a shared library (.so file) which can be linked from Rust:

This command generates two files: libmylib.so (the shared library) and mylib.h (the C header file).

Create a Rust Project

Update the Cargo.toml file to include the libc crate, which provides Rust bindings to C functions and types:

[dependencies]
libc = "0.2"

Write Rust Code to Call GoLang Functions:

extern crate libc;

use libc::{c_int};
use std::ffi::CString;
use std::os::raw::c_char;

#[link(name = "mylib", kind = "dylib")]
extern "C" {
    fn Add(a: c_int, b: c_int) -> c_int;
}

fn main() {
    let a: c_int = 5;
    let b: c_int = 3;
    let result: c_int;
    
    unsafe {
        result = Add(a, b);
    }

    println!("The sum of {} and {} is {}", a, b, result);
}

Here, we use the extern block to declare the Add function, which is implemented in the GoLang library. The #[link(name = "mylib", kind = "dylib")] attribute tells Rust to link against the libmylib.so shared library.

Build and Run the Rust Project:

Before building the Rust project, ensure that the Rust compiler can find the GoLang shared library by setting the LD_LIBRARY_PATH environment variable:

Now, build and run the Rust project:

How it works

1. Compilation and Linking

GoLang Side

1. Go Source Code:
   The Go source code (mylib.go) includes a function Add that is annotated with //export Add, making it accessible from other languages using cgo.

   package main

   import "C"

   //export Add
   func Add(a, b int) int {
       return a + b
   }

   func main() {}

2. Shared Library Generation:
   The Go code is compiled into a shared library (.so file) using the command:

• -buildmode=c-shared: Instructs the Go compiler to create a shared library suitable for linking with C programs.
• libmylib.so: The output file, a shared object library that contains the compiled Add function. Alongside the shared library, a header file (mylib.h) is generated. This header file contains the necessary C declarations that match the exported Go functions, allowing other languages to call them.

Rust Side

1. Rust Source Code:
   The Rust code (main.rs) declares the external Go function and uses it within the Rust program.

   extern crate libc;

   use libc::{c_int};

   #[link(name = "mylib", kind = "dylib")]
   extern "C" {
       fn Add(a: c_int, b: c_int) -> c_int;
   }

   fn main() {
       let a: c_int = 5;
       let b: c_int = 3;
       let result: c_int;

       unsafe {
           result = Add(a, b);
       }

       println!("The sum of {} and {} is {}", a, b, result);
   }

2. Linking:
   The Rust compiler is instructed to link against the libmylib.so shared library through the #[link(name = "mylib", kind = "dylib")] attribute. The extern "C" block specifies that the function Add follows the C calling convention, making it compatible with the Go-compiled shared library.

2. Execution Flow

At Runtime

1. Library Loading:
   When the Rust program is executed, the operating system’s dynamic linker loads the libmylib.so shared library into memory. The LD_LIBRARY_PATH environment variable is set to ensure the shared library can be found.
2. Function Invocation:
   The unsafe block in Rust is necessary because calling an external function involves bypassing Rust’s safety checks. Inside the unsafe block, the Add function is called with two integers, a and b.
3. Cross-language Function Call:

• The Rust code prepares the arguments and calls the Add function.
• The call is passed to the shared library’s entry point for the Add function.
• The Go runtime handles the function call and executes the Go code.
• The result is passed back through the shared library interface to the Rust caller.

1. Result Handling:
   The result returned by the Go function is received by the Rust program and printed to the console.

3. Low-Level Interactions

• Calling Conventions:
  The C calling convention (extern "C") standardizes how functions receive parameters and return values, ensuring compatibility between Rust and Go.
• Memory Management:
  Since Rust and Go have different memory management models, interactions must carefully manage memory to avoid leaks or undefined behavior. In this example, only simple integers are passed, avoiding complex memory management issues.
• Dynamic Linking:
  At runtime, the dynamic linker resolves the symbols (function names) in the shared library, binding them to the corresponding calls in the Rust program. This allows the Rust code to execute the Go function as if it were a native Rust function.

Conclusion

Calling GoLang functions from Rust using FFIs allows developers to harness the strengths of both languages in a single project. This interoperability can be particularly useful in scenarios where different parts of a system are written in different languages, enabling code reuse and performance optimization. By following the steps outlined in this blog, you can start integrating GoLang libraries into your Rust projects, leveraging the unique features of each language to build robust and efficient software.