Bootstrapping LDC 1.26 with GDC 9.3 results in LDC2 binary being linked dynamically with druntime and phobos.

[dbankov-broadcom]

While attempting to bootstrap LDC 1.26 with GDC 9.3 located at a custom location LDC build failed complaining that it couldn't load libgphobos.so.76. Investigating a bit revealed that the link parameters extracted at https://github.com/ldc-developers/ldc/blob/6de8bfde5aa5bc5b528e0771b576281063fcb473/cmake/Modules/ExtractDMDSystemLinker.cmake#L72 from GDMD link line and stored in D_LINKER_ARGS are later passed to the a C++ compiler at https://github.com/ldc-developers/ldc/blob/6de8bfde5aa5bc5b528e0771b576281063fcb473/cmake/Modules/BuildDExecutable.cmake#L77 not being prefixed with "-Wl,". This works fine for "-l.*" and "-L.*" arguments but not for "-B.*" ones. The result is that a "-Bstatic" argument before "-lgphobos" and "-lgdruntime" arguments and a "-Bdynamic" argument after these are simply dropped resulting in both Phobos and DRuntime being linked dynamically with the LDC2 binary. Later during LDC build when LDC2 is used to build some D code it can't load Phobos and DRuntime from GDC 9.3 as it is located at custom location which is not indicated in any way in the environment used with LDC2 and there is no "rpath" in LDC2 pointing to these.

I was able to resolve this problem by prefixing each item from "D_LINKER_ARGS" with "-Wl," which unfortunately revealed another problem.

When Phobos and DRuntime are linked statically to LDC2, the linker complains that there are several duplicate symbols namely "_d_allocmemory", "_d_newclass", "_d_newitemT" and "_d_newitemiT" which appeared to be defined in https://github.com/ldc-developers/ldc/blob/master/dmd/root/rmem.d.

I was able to resolve the second problem by putting the mentioned functions within "version(LDC)" blocks (similar to other functions in this file) but am not sure if this is the correct solution.

I'll post a pull request with the solutions I came up with shortly.

[kinke]

1. You can use -DLDC_LINK_MANUALLY=OFF in the CMake command-line to let the D host compiler perform the linking of the ldc executables, instead of going through the C++ compiler.
2. Wrt. the duplicate symbols, I think that requires a newer GDC host compiler. The line to be patched is https://github.com/ldc-developers/ldc/blob/6de8bfde5aa5bc5b528e0771b576281063fcb473/dmd/root/rmem.d#L209.

[kinke]

Btw, for an optimal LDC build, the preferred bootstrapping way is to build a first version of LDC with some host compiler like GDC, and then use that LDC build to rebuild itself for the final build. This way, shared GDC libs shouldn't matter, and rmem.d doesn't need to be patched, and the resulting compiler should be somewhat faster and consume less RAM without -lowmem.

[dbankov-broadcom]

1. You can use -DLDC_LINK_MANUALLY=OFF

Unfortunately this didn't work for me. It seems that with "LDC_LINK_MANUALLY=OFF" some arguments are prefixed with "-Xcc=" and GDMD that I'm using (https://github.com/D-Programming-GDC/gdmd) reports "gdmd: unrecognized switch '-Xcc=..." for these.

[lenoil98]

I can confirm that the fix provided by @dbankov-vmware works. I’ve used it when building LDC on FreeBSD PowerPC64 when using GDC as Host D. Without this patch, LDC is dependent on libgphobos provided by GDC.

Recommend this fix be merged into master.

[dbankov-broadcom]

Hi @lenoil98,

Nice to hear this worked for you on FreeBSD PowerPC64!

Having said the above I wonder which version of GCC/GDC you have used to bootstrap LDC.

I'm asking because the fix I posted would result in duplicate symbols (for me on Linux) if used with GCC/GDC 9.3 or any GCC/GDC that doesn't support @weak symbols and has the corresponding symbols marked as @weak in GDRuntime. And the latter was introduced somewhat recently in the upstream DRuntime with https://github.com/dlang/druntime/commit/293b45f565e14d838a4604e04ec12a61490d257d.

Based on the above I was with the impression that the fix I've proposed should be merged only after the above change makes it into GCC/GDC and I wondered if this had already happened or there is another reason you didn't observe the duplicate symbols problem.

Best regards, Dentcho Bankov

[dbankov-broadcom]

Hi @lenoil98,

I just realised that my patch avoids the symbols conflict by adding -Wl,-allow-multiple-definition (I'd forgoten about that) which is why it worked for you.

Still I don't think allowing multiple definitions is a good solution to the symbols conflict and should be considered merely a workaround to allow the build to succeed. The problem with the multiple definitions is that the produced LDC compiler will not be optimal and should only be used to build a second version of LDC where this time the conflicting symbols in DRuntime will be "weak" (as this time DRuntime will be coming from LDC). Only if following this procedure LDC will be produced as intended (when using my patch).

Because of the above caveat it may not be acceptable to merge the patch in LDC. Also (unfortunately) it means the LDC that you have produced on FreeBSD PowerPC64 may not be optimal :(.

Maybe if LDC build is extended to support bootstrap builds (the way LLVM does see https://llvm.org/docs/AdvancedBuilds.html) my patch could be modified to be used only if bootstrap build is enabled and to be used only during the first stage of the bootstrapping process.

Anyway I hope the above clarifies things a bit and more importantly why the patch is not merged.

Best regards, Dentcho Bankov

[lenoil98]

@dbankov-vmware I understand completely.

Having said the above I wonder which version of GCC/GDC you have used to bootstrap LDC.

I’m using GCC 11.2 on FreeBSD 13.

The problem with the multiple definitions is that the produced LDC compiler will not be optimal and should only be used to build a second version of LDC where this time the conflicting symbols in DRuntime will be "weak" (as this time DRuntime will be coming from LDC).

I’ve been unsuccessful in building LDC with the one created with your patch. I get the below error:

#0 0x0000000812b17248 llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/usr/local/llvm10/lib/libLLVM-10.so+0x1f93248) #1 0x0000000812b178f4 (/usr/local/llvm10/lib/libLLVM-10.so+0x1f938f4) #2 0x0000000812b14970 llvm::sys::RunSignalHandlers() (/usr/local/llvm10/lib/libLLVM-10.so+0x1f90970) #3 0x0000000812b18030 (/usr/local/llvm10/lib/libLLVM-10.so+0x1f94030) #4 0x0000000815924274 handle_signal /usr/src/lib/libthr/thread/thr_sig.c:303:3

However, now that I know what to do to bootstrap LDC on PowerPC64, I have a working LDC, even if not optimal. For now a less than optimal LDC is more current then the current GCC D frontend. I can now build newer versions of some D libraries I wanted to use.