Linking to static Ginkgo libs using pkg-config does not work

[lahwaacz]

I'm building a custom CUDA solver with Ginkgo static libs using the flags specified in ginkgo.pc and getting this error:

g++ -pthread -L /opt/cuda/lib64/ -o MyApp MyApp.cuo -lstdc++fs -lcudart -ldl -lrt -lz -ltinyxml2 -L /tmp/project/build/ginkgo_Release/lib -lginkgo -lginkgo_device /usr/lib/libhwloc.so -lginkgo_omp /usr/lib/libgomp.so /usr/lib/libpthread.a -lginkgo_cuda /opt/cuda/targets/x86_64-linux/lib/libcudart.so /opt/cuda/targets/x86_64-linux/lib/stubs/libcublas.so /opt/cuda/targets/x86_64-linux/lib/stubs/libcusparse.so /opt/cuda/targets/x86_64-linux/lib/stubs/libcurand.so /opt/cuda/targets/x86_64-linux/lib/stubs/libcufft.so -lginkgo_reference -lginkgo_hip -lginkgo_dpcpp  -lgomp
/usr/bin/ld: /tmp/project/build/ginkgo_Release/lib/libginkgo_cuda.a(executor.cpp.o): in function `std::_Sp_counted_deleter<gko::CudaExecutor*, gko::CudaExecutor::create(int, std::shared_ptr<gko::Executor>, bool, gko::allocation_mode)::{lambda(gko::CudaExecutor*)#1}, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_dispose()':
executor.cpp:(.text+0x41f): undefined reference to `gko::nvidia_device::get_mutex(int)'
/usr/bin/ld: executor.cpp:(.text+0x444): undefined reference to `gko::nvidia_device::get_num_execs(int)'
/usr/bin/ld: /tmp/project/build/ginkgo_Release/lib/libginkgo_cuda.a(executor.cpp.o): in function `gko::CudaExecutor::create(int, std::shared_ptr<gko::Executor>, bool, gko::allocation_mode) [clone .cold]':
executor.cpp:(.text.unlikely+0x3dea): undefined reference to `gko::nvidia_device::get_mutex(int)'
/usr/bin/ld: executor.cpp:(.text.unlikely+0x3e10): undefined reference to `gko::nvidia_device::get_num_execs(int)'
/usr/bin/ld: /tmp/project/build/ginkgo_Release/lib/libginkgo_cuda.a(executor.cpp.o): in function `gko::CudaExecutor::increase_num_execs(unsigned int)':
executor.cpp:(.text+0xa9): undefined reference to `gko::nvidia_device::get_mutex(int)'
/usr/bin/ld: executor.cpp:(.text+0xbf): undefined reference to `gko::nvidia_device::get_num_execs(int)'
/usr/bin/ld: /tmp/project/build/ginkgo_Release/lib/libginkgo_cuda.a(executor.cpp.o): in function `gko::CudaExecutor::decrease_num_execs(unsigned int)':
executor.cpp:(.text+0xf9): undefined reference to `gko::nvidia_device::get_mutex(int)'
/usr/bin/ld: executor.cpp:(.text+0x10f): undefined reference to `gko::nvidia_device::get_num_execs(int)'
/usr/bin/ld: /tmp/project/build/ginkgo_Release/lib/libginkgo_cuda.a(executor.cpp.o): in function `gko::CudaExecutor::get_num_execs(unsigned int)':
executor.cpp:(.text+0x149): undefined reference to `gko::nvidia_device::get_mutex(int)'
/usr/bin/ld: executor.cpp:(.text+0x15f): undefined reference to `gko::nvidia_device::get_num_execs(int)'
/usr/bin/ld: /tmp/project/build/ginkgo_Release/lib/libginkgo_hip.a(executor.cpp.o): in function `gko::HipExecutor::get_num_execs(int)':
executor.cpp:(.text+0xc9): undefined reference to `gko::amd_device::get_mutex(int)'
/usr/bin/ld: executor.cpp:(.text+0xdf): undefined reference to `gko::amd_device::get_num_execs(int)'
collect2: error: ld returned 1 exit status
make: *** [Makefile:216: MyApp] Error 1
make: *** Waiting for unfinished jobs....

The problem is that unlike shared libs, the order of static libs on the command line matters. I had to specify another -lginkgo_device at the end to fix this.

[tcojean]

Thanks for the report! Sadly that part is one we did not test so well so far. We'll fix this.

[yhmtsai]

Hi, @lahwaacz #1109 should fix this issue although it needs to clear some debugging information output and fix some nit. If you could test it on your environment, that will be great. Could you give us some usage example on package config? like grabbing the cflag, library, ldflag, or something else?

[lahwaacz]

@yhmtsai The branch seems to work for me, thanks for working on this.

[yhmtsai]

@lahwaacz thanks for checking it