steebchen
Investigate Cgo implementation
Problem
Right now we download or link the Rust binary as a side-car executable to the Go binary. I think many Go developers would prefer you get a single binary at the end, with the Rust binary linked within the Go binary. Performance would also improve.
Suggested Solution
I'd like to investigate using cgo to link Rust with Go. Here are a few examples that "just work" with go get and go run on your local machine:
- https://github.com/mattn/go-sqlite3
- https://github.com/lithdew/quickjs
The problem with cgo arises when you cross-compile (e.g. compile to Linux from a Mac) to another platform because you don't have the compiler tool-chain of the other platform on your source machine.
I'm starting to come around to the idea that zero-config cross-compilation might not be such a necessary goal.
- Rust's recommended way is to use docker to cross compile: https://github.com/rust-embedded/cross
- Go even has some cgo code in
os/userandnet. They've made these code paths optional using build tags, but recommend you use the cgo version if you can.
The reason it's probably not necessary is that:
- Many developers have docker installed locally these days. You can use docker to cross-compile like Rust does.
- CI is very easy to setup nowadays with Github Actions.
We're already working on "packing" the query engine into the final go binary via #200. It works by putting the query engine into a .go file into the user's project directory, which then gets picked up by the user's go build command. It also uses build constraints to be able to cross compile, so that only one query engine suited for the platform you're targeting is packed into the final binary. It gets unpacked at runtime before anything else runs.
However, the cgo way is most certainly still the better way, but we would need to look into it and it's probably quite a lot of work.
This community repo goprisma implements and solves this entirely. We would just need to maintain it ourselves.