Help with a bug that occurs with dynamic library + static library

[MrcSnm]

In my engine, I have been getting this strange error whenever I use --link-internally.

This looks like a real edge case, but I would love some help to get this fixed. Given that you guys have a lot more experience than me, this could be an obvious mistake that I could not know.

Although this looks like a lot of code, I'm p retty sure I've reduced at least 99% already of my codebase. Also, this looks like some kind of bug interaction between those 2 static libraries while building a dynamic library. test_shared_34.zip

[MrcSnm]

So, I think I found the offending part.

Look into api/source/hip/api/graphics/g2d/g2d_binding.d

module hip.api.graphics.g2d.g2d_binding;
public import hip.api.data.commons;
import hip.api.graphics.color;

version(Have_util) version = ImportSpritesheet;

version(ImportSpritesheet)
{
    public import hip.util.data_structures : Array2D, Array2D_GC;
}

If I remove the public import on ImportSpritesheet, it links.

[MrcSnm]

Update 2: Removing hip.util.string also fixes the error. Update 3: Removing either toString or opEquals from struct String solves the bug too.

[kinke]

What's the error that you are seeing? And only with -link-internally, the MS linker works?

[MrcSnm]

Oh, it looks like it doesn't work on ms linker too.

LLD

lld-link: error: undefined symbol: __declspec(dllimport) Interface for hip.api.data.commons.IHipPreloadable
>>> referenced by C:\Users\Hipreme\AppData\Local\dub\cache\test_shared\~master\build\library-debug-windows-x86_64-ldc_v1.32.0-0F8A0E43E75CA4730B6B6BD5EC36D3CFD79135AB2FC58F7B1F09A06ACB002F0A\test_shared.obj:(ldc.dllimport_relocation)
Error: linking with LLD failed

mslinker

Creating library C:\Users\Hipreme\AppData\Local\dub\cache\test_shared\~master\build\library-debug-windows-x86_64-ldc_v1.32.0-69CC14555271A2E45126C7AC317215B7CB996EDE0EE40C4A59103030CA97D45A\test_shared.lib and object C:\Users\Hipreme\AppData\Local\dub\cache\test_shared\~master\build\library-debug-windows-x86_64-ldc_v1.32.0-69CC14555271A2E45126C7AC317215B7CB996EDE0EE40C4A59103030CA97D45A\test_shared.exp
test_shared.obj : error LNK2019: unresolved external symbol __imp__D3hip3api4data7commons15IHipPreloadable11__InterfaceZ referenced in function ldc.dllimport_relocation
C:\Users\Hipreme\AppData\Local\dub\cache\test_shared\~master\build\library-debug-windows-x86_64-ldc_v1.32.0-69CC14555271A2E45126C7AC317215B7CB996EDE0EE40C4A59103030CA97D45A\test_shared.dll : fatal error LNK1120: 1 unresolved externals
Error: C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.36.32532\bin\HostX64\x64\link.exe failed with status: 1120

[kinke]

Just a guess, but as you've mentioned /WHOLEARCHIVE in the other issue, I guess you are aware that it's required exactly for such static libs compiled with -fvisibility=public, which happens with recent dub by default for static libs ending up in a Windows DLL. Can you try linking the DLL manually with -L/WHOLEARCHIVE:bla.lib for both static libs?

[MrcSnm]

No, it doesn't work in there. Also, you can test right now, there is almost no code to review :) It also looks really related to other thing, which is why I'm finding it really strange.

[MrcSnm]

For example: right now to fix this linker issue

• Remove the import on g2d_binding (located on api) It looks like some interaction between 2 libraries.

Hipengine API does not import util, but it accesses it through a version(Have_util). Which makes it import, even though it is not a dependency. Utils simply exists.

[kinke]

/WHOLEARCHIVE works for me with LDC v1.31. dub build -v with original linking cmdline:

C:\users\packer\dlang\ldc-1.31.0-sym3\bin\ldc2.exe -ofC:\Users\packer\AppData\Local\dub\cache\test_shared\~master\build\library-debug-hDW8YLE1ZVPJ7wLXHHNnIw\test_shared.dll C:\Users\packer\AppData\Local\dub\cache\test_shared\~master\build\library-debug-hDW8YLE1ZVPJ7wLXHHNnIw\test_shared.obj C:\Users\packer\AppData\Local\dub\cache\hipengine_api\~master\build\script-debug-XLe1SoMpflO1lIo_d2vvrg\hipengine_api.lib C:\Users\packer\AppData\Local\dub\cache\util\~master\build\library-debug-bz9ubPFLpKbYegIlMvik0w\util.lib --link-internally -shared -g
lld-link: error: undefined symbol: __declspec(dllimport) Interface for hip.api.data.commons.IHipPreloadable
>>> referenced by C:\Users\packer\AppData\Local\dub\cache\test_shared\~master\build\library-debug-hDW8YLE1ZVPJ7wLXHHNnIw\test_shared.obj:(ldc.dllimport_relocation)
Error: linking with LLD failed

=> fixed up manually by prepending -L/WHOLEARCHIVE: to both static libs:

C:\users\packer\dlang\ldc-1.31.0-sym3\bin\ldc2.exe -ofC:\Users\packer\AppData\Local\dub\cache\test_shared\~master\build\library-debug-hDW8YLE1ZVPJ7wLXHHNnIw\test_shared.dll C:\Users\packer\AppData\Local\dub\cache\test_shared\~master\build\library-debug-hDW8YLE1ZVPJ7wLXHHNnIw\test_shared.obj -L/WHOLEARCHIVE:C:\Users\packer\AppData\Local\dub\cache\hipengine_api\~master\build\script-debug-XLe1SoMpflO1lIo_d2vvrg\hipengine_api.lib -L/WHOLEARCHIVE:C:\Users\packer\AppData\Local\dub\cache\util\~master\build\library-debug-bz9ubPFLpKbYegIlMvik0w\util.lib --link-internally -shared -g
<links fine>

[MrcSnm]

/WHOLEARCHIVE works for me with LDC v1.31. dub build -v with original linking cmdline:

C:\users\packer\dlang\ldc-1.31.0-sym3\bin\ldc2.exe -ofC:\Users\packer\AppData\Local\dub\cache\test_shared\~master\build\library-debug-hDW8YLE1ZVPJ7wLXHHNnIw\test_shared.dll C:\Users\packer\AppData\Local\dub\cache\test_shared\~master\build\library-debug-hDW8YLE1ZVPJ7wLXHHNnIw\test_shared.obj C:\Users\packer\AppData\Local\dub\cache\hipengine_api\~master\build\script-debug-XLe1SoMpflO1lIo_d2vvrg\hipengine_api.lib C:\Users\packer\AppData\Local\dub\cache\util\~master\build\library-debug-bz9ubPFLpKbYegIlMvik0w\util.lib --link-internally -shared -g
lld-link: error: undefined symbol: __declspec(dllimport) Interface for hip.api.data.commons.IHipPreloadable
>>> referenced by C:\Users\packer\AppData\Local\dub\cache\test_shared\~master\build\library-debug-hDW8YLE1ZVPJ7wLXHHNnIw\test_shared.obj:(ldc.dllimport_relocation)
Error: linking with LLD failed

=> fixed up manually by prepending -L/WHOLEARCHIVE: to both static libs:

C:\users\packer\dlang\ldc-1.31.0-sym3\bin\ldc2.exe -ofC:\Users\packer\AppData\Local\dub\cache\test_shared\~master\build\library-debug-hDW8YLE1ZVPJ7wLXHHNnIw\test_shared.dll C:\Users\packer\AppData\Local\dub\cache\test_shared\~master\build\library-debug-hDW8YLE1ZVPJ7wLXHHNnIw\test_shared.obj -L/WHOLEARCHIVE:C:\Users\packer\AppData\Local\dub\cache\hipengine_api\~master\build\script-debug-XLe1SoMpflO1lIo_d2vvrg\hipengine_api.lib -L/WHOLEARCHIVE:C:\Users\packer\AppData\Local\dub\cache\util\~master\build\library-debug-bz9ubPFLpKbYegIlMvik0w\util.lib --link-internally -shared -g
<links fine>

I have tested, this approach works with dub on msvc linker, but not with --link-internally, because it can't find the libraries

[MrcSnm]

Should then I report this bug as dub's bug or ldc then?

[kinke]

I don't think we can sensibly handle this in the compiler, without making working with Windows DLLs a similar pain as with C++. So I think this should be handled on the build-system level, incl. dub. But needs to be opt-out, not applied to every static lib being linked into a Windows DLL.