C FFI

[LightAndLight]

When it comes to building a standard library, I'll probably need some sort of C FFI. At the very least, I need to be able to use Linux system calls. The alternative is hard-coding each syscall into the interpreter as a primitive. I think this will get in the way of scaling library functionality.

There's a good intro to FFIs for interpreted languages on the Inko blog. That page mentions libffi as a possible library for implementing FFIs.

It looks like the way to go is to use shared libraries via dlopen and dlsym (Rust wrapper here). I can use dlopen to obtain a function pointer of type const * std::ffi::c_void. The big question is how I can use that function pointer as if it were an ipso function.

The Inko blog mentions that using dlopen means you can't use musl libc. While investigating this, I found this mailing list post about why having a statically linked binary that uses dlopen is a bad idea. The gist is that if your static binary uses a different version of libc to the library that you dlopen, then bad things can happen.

So, what can we do about this?

1. Ignore it, compile ipso as a dynamically-linked executable, and forgo static compilation and musl support

   I have a feeling that I want the ipso binary to allow for static compilation, because it might make it easier to adopt. The problems laid out the the musl mailing list post seem worth avoiding, though.

   I also might like the executable to be as small as possible, for which dynamic linking is a good option.

2. Use traditional compilation under the hood, with normal static or dynamic linking

   This means that the project will no longer be an interpreter, and will look more like a JIT-compiled language. This course of action would probably result in actual JIT compilation. But if we go full JIT, I think we run into the same problems because the JIT library still needs to locate the function pointers for the various functions.

In the interest of moving forward, I think the right choice is to figure out the minimum required C-based functionality, and implement them as primitives. Then I can revisit FFI when this design choice is limiting my ability to scale out the standard library.