mold 1.1 failed to find libraries with gcc 11.2.0 on riscv64

[xctan]

Currently gcc (version 11.2.0) behaves differently on x86-64 and riscv64 platforms. The riscv64 version of gcc doesn't pass enough search dirs (such as /lib, /usr/lib) to mold, resulting in library resolution failure.

Tested on Arch Linux RISC-V:

[root@archriscv2 tmp]# uname -a
Linux archriscv2 5.16.9-arch1-1 #1 SMP PREEMPT Fri, 11 Feb 2022 22:42:06 +0000 riscv64 GNU/Linux
[root@archriscv2 tmp]# pacman -Qe gcc
gcc 11.2.0-3
[root@archriscv2 tmp]# echo 'int main() { return 0; }' > b.c
[root@archriscv2 tmp]# mold -run gcc -o ./exe b.c --verbose
  ......
LIBRARY_PATH=/usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-o' './exe' '-v' '-march=rv64imafdc' '-mabi=lp64d' '-march=rv64imafdc' '-dumpdir' './exe.'  /usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/collect2 -plugin /usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/liblto_plugin.so -plugin-opt=/usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/lto-wrapper -plugin-opt=-fresolution=/tmp/ccCsaMzi.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --build-id --eh-frame-hdr --hash-style=gnu -melf64lriscv -dynamic-linker /lib/ld-linux-riscv64-lp64d.so.1 -pie -o ./exe /lib/Scrt1.o /usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/crti.o /usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/crtbeginS.o -L/usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0 /tmp/cc5h1e2O.o -lgcc --push-state --as-needed -lgcc_s --pop-state -lc -lgcc --push-state --as-needed -lgcc_s --pop-state /usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/crtendS.o /usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/crtn.o
mold: library not found: gcc_s
collect2: error: ld returned 1 exit status

gcc works well with bfd:

[root@archriscv2 tmp]# gcc -o ./exe b.c --verbose -fuse-ld=bfd
  ......
COLLECT_GCC_OPTIONS='-o' './exe' '-v' '-fuse-ld=bfd' '-march=rv64imafdc' '-mabi=lp64d' '-march=rv64imafdc' '-dumpdir' './exe-'         as -v --traditional-format -fpic -march=rv64imafdc -march=rv64imafdc -mabi=lp64d -o /tmp/ccS4vr66.o /tmp/ccs7PEig.s
GNU assembler version 2.36.1 (riscv64-unknown-linux-gnu) using BFD version (GNU Binutils) 2.36.1
COMPILER_PATH=/usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/:/usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/:/usr/lib/gcc/riscv64-unknown-linux-gnu/:/usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/:/usr/lib/gcc/riscv64-unknown-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-o' './exe' '-v' '-fuse-ld=bfd' '-march=rv64imafdc' '-mabi=lp64d' '-march=rv64imafdc' '-dumpdir' './exe.'         /usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/collect2 -plugin /usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/liblto_plugin.so -plugin-opt=/usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/lto-wrapper -plugin-opt=-fresolution=/tmp/ccDwG9a0.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --build-id --eh-frame-hdr --hash-style=gnu -melf64lriscv -dynamic-linker /lib/ld-linux-riscv64-lp64d.so.1 -pie -fuse-ld=bfd -o ./exe /lib/Scrt1.o /usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/crti.o /usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/crtbeginS.o -L/usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0 /tmp/ccS4vr66.o -lgcc --push-state --as-needed -lgcc_s --pop-state -lc -lgcc --push-state --as-needed -lgcc_s --pop-state /usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/crtendS.o /usr/lib/gcc/riscv64-unknown-linux-gnu/11.2.0/crtn.o COLLECT_GCC_OPTIONS='-o' './exe' '-v' '-fuse-ld=bfd' '-march=rv64imafdc' '-mabi=lp64d' '-march=rv64imafdc' '-dumpdir' './exe.'     

But with clang 13.0.1, mold works correctly because clang does keep library search path full:

[root@archriscv2 tmp]# mold -run clang -o ./exe b.c --verbose
  ......
 "/usr/sbin/ld" -pie --eh-frame-hdr -m elf64lriscv -dynamic-linker /lib/ld-linux-riscv64-lp64d.so.1 -o ./exe /usr/bin/../lib/Scrt1.o /usr/sbin/../lib/gcc/riscv64-unknown-linux-gnu/11.2.0/crti.o /usr/sbin/../lib/gcc/riscv64-unknown-linux-gnu/11.2.0/crtbeginS.o -L/usr/sbin/../lib/gcc/riscv64-unknown-linux-gnu/11.2.0 -L/usr/bin/../lib -L/lib -L/usr/lib /tmp/b-f90d0c.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/sbin/../lib/gcc/riscv64-unknown-linux-gnu/11.2.0/crtendS.o /usr/sbin/../lib/gcc/riscv64-unknown-linux-gnu/11.2.0/crtn.o

I've found that mold will add default library paths for macho files when nostdlib is not presented. Maybe we need to do the same thing on riscv64 platform?

[rui314]

On all the other platforms I know of, gcc always passes all paths required for linking, so I think this is gcc's bug.

mold (and LLVM lld) do not have the notion of "default search paths" by design. They only relies on command line arguments, so as long as you pass the exact same arguments, they are guaranteed to produce the same results. That makes cross compilation pretty easy.

[rui314]

I reported this issue to gcc: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104708

[jpalus]

I have the same issue on ARM32 and can't really tell who should fix it. It appears that gcc does not pass /lib and /usr/lib down to linker on purpose since it "Excludes directories that the linker is known to search" (source).

I suppose either mold should hardcode searching in /lib /usr/lib or gcc should drop this check.

[rui314]

We shouldn't add the notion of "default search paths" to mold for the sake of build reproducibility. Currently, it is guaranteed that mold produces the exact same output as long as you pass the same command line options and the same input files. That reproducibility is guaranteed across platforms, which makes cross building easy.

We should fix GCC instead.