Feature request: drop dependency on host /bin/sh

[bjornfor]

I think it would be cool if the dependency on host /bin/sh could be dropped. Then nix-bundle would truly have no external dependencies.

Perhaps the entry point could be something statically linked with musl?

Currently the front-page says "No external dependencies are required besides a compatible Linux kernel". So, to truly live up to that, it shouldn't use host /bin/sh :-)

[matthewbauer]

Yeah, this is a good point. I guess in my opinion "shell script" is a pretty versatile format in the same way that "ELF" is. I'll change that description for now. I've always hoped to get this working on macOS as well but it's not easy getting a "chroot"-like thing in Mach kernel.

Anyway the "Arx" format is made to use shell scripts so we'd have to switch to something else. AppImageKit seems like the best alternative and I've tried getting something to work but it's not quite working yet.

[matthewbauer]

Actually, does /bin/sh not exist on NixOS? We can try to switch it to /usr/bin/env sh if so.

[bjornfor]

NixOS has /bin/sh.

[bjornfor]

I found more undeclared dependencies. The resulting file from ./nix-bundle.sh hello /bin/hello depends not only on "/bin/sh" but also "date", "tr", and "hexdump".

[matthewbauer]

"date", "tr", and "hexdump"

So those are all used for the "self-executing" bootstrapping through arx. They have been available in every Linux distro I've tried so far. I know date and tr are part of GNU coreutils although I'm not sure what provides hexdump?

[bjornfor]

hexdump comes from util-linux.

Any ideas where to start replacing shell + Arx with some C code we can statically link with musl? I was hoping that once the "entry point" logic was dealt with, the code could simply run "ld-linux.so_from_bundle application_from_bundle".

[bjornfor]

Hm, AppImageKit seems to be building several libraries statically (libsquashfs, liblzma). Perhaps it's not that much work left to make the entry point static too? (Of course, we'd still need Nix to create the inputs that goes into the AppDir that gets wrapped into this AppImage image format.)

[probonopd]

Perhaps it's not that much work left to make the entry point static too?

What would be the advantage of that?

[bjornfor]

@probonopd: In the case of AppImage, not using static entry point leads to this (on NixOS):

$ ./Subsurface-4.6.2-x86_64.AppImage 
bash: ./Subsurface-4.6.2-x86_64.AppImage: No such file or directory

$ file ./Subsurface-4.6.2-x86_64.AppImage 
./Subsurface-4.6.2-x86_64.AppImage: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=783ad4749a42b2e09fedee6b1b69accd9e010fe4, stripped

$ nix-build -A glibc '<nixpkgs>'
/nix/store/kjwbqnh13dxh6w4pk2gb3ddmhpiaihqg-glibc-2.25

$ /nix/store/kjwbqnh13dxh6w4pk2gb3ddmhpiaihqg-glibc-2.25/lib/ld-linux-x86-64.so.2 ./Subsurface-4.6.2-x86_64.AppImage
./Subsurface-4.6.2-x86_64.AppImage: error while loading shared libraries: ./Subsurface-4.6.2-x86_64.AppImage: ELF file ABI version invalid

The first problem is that the file refers to the system linker at /lib/... The next problem is that my system linker (when explicitly invoked) turns out to be incompatible with the rest of this AppImage (glibc presumably).

nix-bundle doesn't have that issue, because it bundles the dynamic linker. However, it relies on /bin/sh and a few other command-line tools. Although those tools are very common to have, I think there is something neat about a package that only relies on a the kernel interface, and nothing from userspace. (The kernel interface is much more stable than userspace.)

[probonopd]

I think it's a NixOS issue that "normal" Linux executables can't be run unless one manually puts a linker into the usual place.

[bjornfor]

@probonopd: True, but even if NixOS would provide a /lib64/lib/ld-linux-x86-64.so.2, the problem of version incompatibilities would still exist.

[bobvanderlinden]

I ran into this problem just today:

$ NIX_PATH="nixpkgs=https://github.com/nixos/nixpkgs/archive/release-18.03.tar.gz" result/bin/nix-bundle curl /bin/curl
$ ./curl
./curl: line 6: hexdump: command not found

[immae]

you can add tar and bzip2 to the host-dependency list (at the end of the script). tar might be standard, but bzip2 is not

[bjornfor]

It looks like https://github.com/ralismark/nix-appimage provides a static entrypoint (using musl), essentially solving this issue.