LDC is trying to reference ModuleInfo from `ldc.gccbuiltins` and friends

[ljmf00]

trafficstars

CC @JohanEngelen

ldc.gccbuiltins_ and other intrinsics modules are effectively a bunch of forward declarations with pragma(LDC_intrinsic) which doesn't require ModuleInfo.

I can't get a small reproduction of this, but some module in our codebase, if get imported, generates references to it, which I believe is the compiler trying to ensure some static ctor order, based on its imports.

ld.lld: error: undefined symbol: ModuleInfo for ldc.gccbuiltins_x86
>>> referenced by package.d
>>>               ../weka-lib_Dsit.o:(ModuleInfo for weka.lib.bitmanip_ext)

ld.lld: error: undefined symbol: ModuleInfo for ldc.llvmasm
>>> referenced by package.d
>>>               ../weka-lib_Dsit.o:(ModuleInfo for weka.lib.compiler.asm_utils)
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)

For such modules, pragma(LDC_no_module_info) should be used and the compiler should prevent referencing modules that have that pragma.

[JohanEngelen]

Good one. These modules are not included when building druntime/phobos (thus are not part of any user's build/link), so indeed pragma(LDC_no_module_info) seems the right thing to do.

[kinke]

AFAIK, a ModuleInfo A only references another imported module B if B has a module ctor directly, or imports (directly or indirectly) another module with a module ctor (as said, for proper module ctor order). IIRC, I've also seen similar cases at work, with .di files not importing anything at all. Now I'm not sure if there's a (DMD) bug in there, or whether e.g. the new templated druntime hooks or general druntime (incl. object.d!) refactorings caused hidden import graphs involving some module with a module ctor.

[ljmf00]

I didn't thought about that last possibility. They seem to not import anything relevant regarding static ctors, at least the affecting modules, in my case. The fact that I don't have a reproducible way of testing this makes it difficult to run a regression test against DMD frontend and runtime with the new templated druntime hooks.

In the other way around, I don't think its sensible to include a stub ModuleInfo for these issues but it could be an alternative solution. Is .di files supposed to generate ModuleInfo at all? They assume the existance of ModuleInfo, right?

[kinke]

Is .di files supposed to generate ModuleInfo at all?

Nope. They aren't compiled, so pragma(LDC_no_moduleinfo); currently doesn't affect them at all.

The pragma is ignored for the referencing part - and I like that, if working as intended. I.e., you would previously only get such errors (for .di headers, but also potentially across DLL boundaries on Windows) if the module being imported did depend on some module ctor, so it's needed for a correct module ctor execution order.

But if that's a problem / buggy now, we might want to avoid such ModuleInfo.importedModules refs for modules with that pragma and .di headers in general - preferably with a warning or message that the module ctor execution order might be off. Edit: That's in https://github.com/ldc-developers/ldc/blob/d4f2bed3a45688b32434d429e39c22dea5a9ffda/gen/moduleinfo.cpp#L143-L146 (mod->noModuleInfo)

[kinke]

Yeah well doing this for all .di headers is probably not desirable. E.g., some vendor might distribute prebuilt libs and .di stubs for the interface; in that case, the .di ModuleInfos would be linkable from the prebuilt libs. (But making sure the .di has the same imports as the actual thing is probably another problem.)

[anon17]

If nonreproducible means nondeterministic, you can replace the compilation command with one that tries to compile many times until failure.

[anon17]

Or it depends on compilation order. Do you use incremental compilation?

[ljmf00]

If nonreproducible means nondeterministic, you can replace the compilation command with one that tries to compile many times until failure.

It is deterministic. Its just a too complex reproduction to narrow down to a simple testcase, because its related to imports and how they reference each other. @JohanEngelen have access to the codebase, I can tell him the reproduction steps for the issue. Would be cool indeed to know the core issue, I think it might be related to some druntime, as mentioned by @kinke , but can't be sure.

[kinke]

With what LDC version were these problems encountered? At work, we hit one such instance with D v2.101, but D v2.102 (LDC v1.32) fixed it.

[JohanEngelen]

With what LDC version were these problems encountered?

Weka is using 1.30 at the moment.

[ljmf00]

With what LDC version were these problems encountered? At work, we hit one such instance with D v2.101, but D v2.102 (LDC v1.32) fixed it.

We cant check this until we upgrade it. Will feedback when we do, I guess. Idk how is the progress there @JohanEngelen but I got hitten by lot of dip1000 issues. I've been fixing those as I encounter them.