mold

mold copied to clipboard

It appears that object merging when using a linker script produces object files with no symbols.

> tree .
.
├── ghc_14.ldscript
├── input.o
├── lib.o
├── mold-cmd
└── out.dyn_o

> cat ghc_14.ldscript
INPUT("lib.o" "input.o")

[nix-shell:~/ghc-scratch/mold-bug]$ nm lib.o 
                 U registerForeignExports
0000000000000000 b stg_exports_GHCziCString
0000000000000000 t stginit_export_GHCziCString

[nix-shell:~/ghc-scratch/mold-bug]$ nm input.o 
0000000000000000 D ghczmprim_GHCziCString_Bytes_closure_tbl
0000000000000230 D ghczmprim_GHCziCString_cstringLengthzh_closure
0000000000000fb8 T ghczmprim_GHCziCString_cstringLengthzh_info
0000000000000228 D ghczmprim_GHCziCString_czustrlen_closure
0000000000000f88 T ghczmprim_GHCziCString_czustrlen_info
0000000000000250 D ghczmprim_GHCziCString_Four_closure
0000000000001038 T ghczmprim_GHCziCString_Four_con_info
0000000000000238 D ghczmprim_GHCziCString_One_closure
0000000000000fd8 T ghczmprim_GHCziCString_One_con_info
0000000000000248 D ghczmprim_GHCziCString_Three_closure
0000000000001018 T ghczmprim_GHCziCString_Three_con_info
0000000000000240 D ghczmprim_GHCziCString_Two_closure
0000000000000ff8 T ghczmprim_GHCziCString_Two_con_info
0000000000000028 D ghczmprim_GHCziCString_unpackAppendCStringUtf8zh_closure
00000000000009d0 T ghczmprim_GHCziCString_unpackAppendCStringUtf8zh_info
0000000000000008 D ghczmprim_GHCziCString_unpackAppendCStringzh_closure
0000000000000180 T ghczmprim_GHCziCString_unpackAppendCStringzh_info
0000000000000030 D ghczmprim_GHCziCString_unpackCStringUtf8zh_closure
0000000000000d40 T ghczmprim_GHCziCString_unpackCStringUtf8zh_info
0000000000000000 D ghczmprim_GHCziCString_unpackCStringzh_closure
0000000000000058 T ghczmprim_GHCziCString_unpackCStringzh_info
0000000000000020 D ghczmprim_GHCziCString_unpackFoldrCStringUtf8zh_closure
0000000000000858 T ghczmprim_GHCziCString_unpackFoldrCStringUtf8zh_info
0000000000000010 D ghczmprim_GHCziCString_unpackFoldrCStringzh_closure
00000000000003b0 T ghczmprim_GHCziCString_unpackFoldrCStringzh_info
0000000000000018 D ghczmprim_GHCziCString_unpackNByteszh_closure
0000000000000408 T ghczmprim_GHCziCString_unpackNByteszh_info
0000000000000078 D ghczmprim_GHCziCString_zdtcBytes1_closure
0000000000000017 R ghczmprim_GHCziCString_zdtcBytes2_bytes
0000000000000088 D ghczmprim_GHCziCString_zdtcBytes_closure
00000000000000c8 D ghczmprim_GHCziCString_zdtczqFour1_closure
00000000000001d8 D ghczmprim_GHCziCString_zdtczqFour2_closure
0000000000000000 R ghczmprim_GHCziCString_zdtczqFour3_bytes
00000000000001e8 D ghczmprim_GHCziCString_zdtczqFour_closure
00000000000000e8 D ghczmprim_GHCziCString_zdtczqOne1_closure
0000000000000012 R ghczmprim_GHCziCString_zdtczqOne2_bytes
00000000000000f8 D ghczmprim_GHCziCString_zdtczqOne_closure
0000000000000188 D ghczmprim_GHCziCString_zdtczqThree1_closure
0000000000000006 R ghczmprim_GHCziCString_zdtczqThree2_bytes
0000000000000198 D ghczmprim_GHCziCString_zdtczqThree_closure
0000000000000138 D ghczmprim_GHCziCString_zdtczqTwo1_closure
000000000000000d R ghczmprim_GHCziCString_zdtczqTwo2_bytes
0000000000000148 D ghczmprim_GHCziCString_zdtczqTwo_closure
0000000000000048 D ghczmprim_GHCziCString_zdtrModule1_closure
000000000000001d R ghczmprim_GHCziCString_zdtrModule2_bytes
0000000000000038 D ghczmprim_GHCziCString_zdtrModule3_closure
0000000000000029 R ghczmprim_GHCziCString_zdtrModule4_bytes
0000000000000058 D ghczmprim_GHCziCString_zdtrModule_closure
                 U ghczmprim_GHCziTypes_Czh_con_info
                 U ghczmprim_GHCziTypes_KindRepTyConApp_con_info
                 U ghczmprim_GHCziTypes_krepzdzt_closure
                 U ghczmprim_GHCziTypes_Module_con_info
                 U ghczmprim_GHCziTypes_TrNameS_con_info
                 U ghczmprim_GHCziTypes_TyCon_con_info
                 U ghczmprim_GHCziTypes_ZC_con_info
                 U ghczmprim_GHCziTypes_ZMZN_closure
000000000000004b r i15E_str
0000000000000064 r i15J_str
000000000000007f r i15O_str
0000000000000032 r i15z_str
                 U stg_ap_0_fast
                 U stg_ap_pp_fast
                 U stg_gc_noregs
                 U stg_gc_unbx_r1
                 U stg_upd_frame_info
                 U strlen

Command:

mold -r '--build-id=none' -o out.dyn_o ghc_14.ldscript

Expected output, an object file which contains symbols from both input object files.

Actual output, an object file with no symbols in.

[nix-shell:~/ghc-scratch/mold-bug]$ nm out.dyn_o 
nm: out.dyn_o: no symbols

Repro: mold-bug.tar.gz

Note that if I don't use a linker script and pass the file names directly on the command line then the merge works.

Linker script doesn't work if you pass -r. We can fix that, but is it a realistic use case? Could you tell me a bit more about how you find this problem?

I found this problem because GHC uses linker scripts in order to avoid running into long command line issues.

For what it's worth, GHC uses -r to produce so-called "GHCi objects", which we load into our REPL. We use -r to merge objects' text sections (since the objects in question may have been built with function-sections and therefore may have many sections, greatly increasing the effort needed to load them).

mold's -r does not merge multiple .text sections into one .text. It emits the same number of .text sections as they are in the input files.

Maybe a silly question, but why GHC doesn't use .so instead of .o?

Maybe a silly question, but why GHC doesn't use .so instead of .o?

It's not a silly question at all. GHC in fact does support building shared objects and most users do rely on this functionality. However, it's hard to fully support all GHC features due to limitations of existing dynamic linkers. There is a (slightly-dated) page on our Wiki which tries to capture the trade-offs but I will try to summarise here.

Code unloading

Specifically, we support unloading and reloading of code (used by our REPL, ghci, as well as compile-time evaluation) and have a garbage collected runtime. Safely supporting this when with most dynamic linkers is at best flaky and platform-dependent and in the worst case impossible.

Concretely, the POSIX dynamic linking interfaces provide no ability to track dependencies between loaded objects, nor any way to enumerate their symbols; both of these things are necessary to determine object liveness in a garbage-collected setting. It happens that glibc does provide some extensions which give us some insight into the structure of loaded objects, but even then we have no control over symbol resolution, which makes code reloading a fraught process (since a symbol from an old version of the reloaded code may still be visible).

Linking performance

GHC uses an optimisation known as info-tables-next-to-code to optimise access patterns for its garbage collector. To be sound, this technique requires that all dynamic symbol references are handled via the GOT (since a function symbol is used both as a code pointer --- to jump to the function's entrypoint --- and as a data pointer --- to access its so-called "info table"). This means that the usual lazy binding logic which most linkers provide is unusable to us. For this reason, we

Runtime performance

Haskell packages tend to be small and interpackage references are fairly common. Unfortunately, we have noticed that the overhead of symbol resolution via the GOT ends up incurring a non-trivial runtime cost.

Thank you for the detailed explanation. So from your comment, I think we need to do two things to make mold work well with ghc. First, we need to fix this bug. Second, we need to make mold to merge .text sections into a single large .text section for the mold -r use case. Both aren't trivial, but I'm happy to do them.

This issue should be fixed by now.