glibc version incompatibilities

[nopper]

I am trying to bootstrap an archlinux system but I keep receiving the following error. The error still remains if I enable the -s flag of fakechroot:

/usr/bin/pacman: relocation error: /tmp/target/usr/lib/libc.so.6: symbol _dl_find_dso_for_object, version GLIBC_PRIVATE not defined in file ld-linux-x86-64.so.2 with link time reference

Do you have any suggestions?

[abliss]

In the "Limitations" section of the manpage, it says: https://github.com/dex4er/fakechroot/blob/master/man/fakechroot.pod#limitations

• /lib/ld-linux.so.2 and /lib64/ld-linux-x86-64.so.2 are always loaded from real environment. This path is hardcoded by linker for all binaries. You can set the FAKECHROOT_ELFLOADER environment variable or use --elfloader option.
• Every command executed within fakechroot needs to be linked to the same version of the dynamic linker from real environment. If the libraries in chroot are not compatible, try to use --use-system-libs option.

So, in short, I think you're out of luck.

[josch]

Hi, I'm the author of a tool called mmdebstrap which is a tool like debootstrap but uses apt doesn't require superuser privileges: https://packages.debian.org/sid/mmdebstrap

I am also running into the problem described here. Notably, I'm getting segmentation fault when I try to use fakechroot with a Debian stable system while I'm on unstable (so the libc differs in version).

I am aware of the limitations from the manpage but I'm wondering if there is maybe a way to autodetect this kind of problem, so that the user gets more explanation than just a "segmentation fault" or which allows to abort early if a certain condition is met. What condition would need checking?

Thanks!

[abliss]

Maybe fakechroot could inspect the file at the path given by its argument, and check if it's an ELF file, and if so, inspect its headers. For example, on my system, readelf -a /bin/ls reports [Requesting program interpreter: /lib/ld-linux-aarch64.so.1] and objdump -x /bin/ls reports:

Version References:
  required from ld-linux-aarch64.so.1:
    0x06969197 0x00 03 GLIBC_2.17

So, it ought to be possible to check whether the required version symbol is present in the host environment's dynamic loader before execing the binary. But I wouldn't expect this to become the default behavior, since that might break existing users who may be managing to use mismatched versions successfully. (Maybe a warning printed to stderr before the exec would be okay, though it'll still probably break somebody somehow.)

Slightly more complicated would be if the named file is not an ELF executable but a text file with a shebang line (or any other magic bytes registered with binfmt_misc?) . Then fakechroot would have to parse out the path for the interpreter and see if that is an ELF program with mismatched ld version symbols.

[abliss]

@josch can you provide exact instructions to reprduce the segfault?

[josch]

@abliss sure! In Debian unstable, create a Debian buster (stable) chroot and then try running anything in it. Like this:

$ sudo debootstrap --variant=minbase buster debian-buster
$ fakechroot fakeroot chroot debian-buster echo foobar
/usr/sbin/chroot.fakechroot: line 105: 15511 Segmentation fault      env -u FAKECHROOT_BASE_ORIG FAKECHROOT_CMD_ORIG= LD_LIBRARY_PATH="$fakechroot_chroot_paths" FAKECHROOT_BASE="$fakechroot_chroot_base" "$fakechroot_chroot_chroot" "${@:1:$(($fakechroot_chroot_n - 1))}" "${fakechroot_chroot_newroot#$FAKECHROOT_BASE_ORIG}" "${@:$(($fakechroot_chroot_n + 1))}"

EDIT: Some more details:

The version of libc6 on my system is 2.29 while the version inside the chroot is 2.28. If I only upgrade the libc6 package (and no other package) inside the chroot from 2.28 to 2.29, then fakechroot does not segfault at all.

[abliss]

Hmm, that seems weird to me. I would think that using ld-linux from libc6 version 2.29 ought to work with the version-2.28 libraries for almost all purposes. Does it segfault with all binaries or just /bin/echo?

[josch]

Lets find out!

for b in debian-buster/bin/* debian-buster/sbin/* debian-buster/usr/bin/* debian-buster/usr/sbin/*; do
    bn=$(basename $b)
    fakechroot fakeroot chroot debian-buster $bn
    if [ $? -ne 139 ]; then
        echo "no segfault">&2
        break
    fi
done

Nope, everything segfaults.

[abliss]

Interesting... what about with --use-system-libs?

[josch]

If in the loop above I instead run fakechroot --use-system-libs fakeroot chroot debian-buster $bn then I still get segmentation faults for all binaries.

[abliss]

Hmm... can you check (perhaps with strace -f) exactly which binary is segfaulting? Is it perhaps fakeroot or chroot itself, rather than the target binary?

[josch]

fakeroot is also just a shell script that adds a few more environment variables and does the LD_PRELOAD magic, so no binary to segfault.

The actual binary that segfaults is env as it is called from /usr/sbin/chroot.fakechroot which is used instead of /usr/sbin/chroot. At the bottom where the script calls env, I prefixed that with gdb --batch -ex=r -ex=bt --args and got you the following backtrace:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7d9b127 in __libc_start_main () from /home/josch/git/mmdebstrap/debian-buster/lib/x86_64-linux-gnu/libc.so.6
#0  0x00007ffff7d9b127 in __libc_start_main () from /home/josch/git/mmdebstrap/debian-buster/lib/x86_64-linux-gnu/libc.so.6
#1  0x0000555555556bda in ?? ()

I cannot give you a trace with debugging symbols because the outside system has a newer libc6 package and the debugging symbols I installed for it are obviously not valid for the older libc6 inside the chroot.

[abliss]

Now we're getting somewhere... which env is being run (and segfaulting), the outside one or the inside one? It looks like the segfault is in the inside libc. As I understand it, libc is (usually) backwards-compatible but not forwards-compatible. So if you're somehow running your outside (newer) env binary against the inside (older) libc library, a segfault is expected. But in normal usage that shouldn't happen. Is there any chance that the chroot's env is on your $PATH earlier than your system's env?

(Does the order of fakeroot and fakechroot in your command matter?)

[josch]

Okay, so we are talking about this script: https://sources.debian.org/src/fakechroot/2.19-3.2/scripts/chroot.fakechroot.sh/

In line 97 it executes env. That's the env from the outside. We can reproduce the problem by replacing env with /usr/bin/env.

If I print my $PATH in the line above, then I see nothing surprising: /usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/sbin:/sbin.

You can easily reproduce my problem in a Debian chroot. Even if you are not using Debian, your distro should contain the debootstrap tool with which you can create such a chroot.

No, the order of fakeroot and fakechroot does not matter. As instructed by the docs, I'm running fakeroot inside fakechroot.

[RadFromOrange]

Hello @josch ,i have same issue 3 years laters.. any news ?

Thx,

[josch]

@RadFromOrange No. I accepted to live with the limitation that the libc in the chroot must be the same as the one on the outside.

[RadFromOrange]

Hello @josch ,i have same issue 3 years laters.. any news ?

Thx,

Ok thanks ,you re saving me days, may be weeks of research time as i don't think it's possible to emulate all features of the real chroot command so easily.