bob icon indicating copy to clipboard operation
bob copied to clipboard

Go binaries built with CGO_ENABLED=1 do not work through nix

Open rdnt opened this issue 2 years ago • 2 comments

Dynamically linked go binaries built inside bob with nix are not executable under different environments (e.g. inside a docker container on the same system).

file ./server

./server: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /nix/store/2j8jqmnd9l7plihhf713yf291c9vyqjm-glibc-2.35-224/lib/ld-linux-x86-64.so.2, Go BuildID=ESlrWFpDn31BIxzZZX0h/0i-We6XI5_Rb-pGBXFbh/lRcJKtpA40xmRxkHsnUm/u7T_cAmfqXS1MDeQwScw, with debug_info, not stripped


CGO_ENABLED=0 is required, I presume, for all Go binaries created through bob. We should consider adding it to the docs, and investigate the underlying problem.

rdnt avatar Apr 04 '23 14:04 rdnt

The problem is that go from nix links against a glibc inside the /nix/store which is not present on systems without nix. Must be fixed in the go compiler packages provided by nix.

Equanox avatar Apr 05 '23 11:04 Equanox

Yeah this is a basic fact of shared libraries.

Nix writes the RPATH in the binaries to point to the nix store version of the shared library. If you then copy that binary somewhere else, the RPATH is pointing somewhere that probably doesn't exist.

Any time you use shared libraries you run a bit of this risk. Non-Nix binaries usually end up relying on ldconfig to specify locations to automatically search for the matching shared library (matching the indicated version of the library). When you copy that binary between systems you're just hoping the system you copied it to has a shared library with the same version that's actually compatible with what the binary needs. One of the things Nix seeks to resolve is this "hoping" problem, so instead it requires a specific deterministic shared library (not just one with a matching version number), and it knows that only the /nix/store/ is going to have an index of libraries based on the deterministic numbering scheme. This is because Nix had to invent their own way to creating a deterministic version number that's more exact than the Semantic Versioning scheme used by general shared libraries and binaries.

I believe there's a way to copy a nix closure to another system, but I don't remember what the command is or what the limitations of it are.

mtalexan avatar Mar 21 '24 16:03 mtalexan