kokkos-kernels icon indicating copy to clipboard operation
kokkos-kernels copied to clipboard

Published 20 hours ago • verified publisher icon kokkos

linker/compiler error during build

Open glhenni opened this issue 10 months ago • 13 comments

About a week or so ago the project I work on started getting errors (sample below) during the build when building kokkoskernels as contained in github:Trilinos/develop. Any idea as to what's going on?

/projects/cde/v3/cee/spack/opt/spack/linux-rhel7-x86_64/gcc-10.3.0/openmpi-4.1.2-ated23f2cr5tikdwru4hsr7pl25jk2bp/bin/mpicxx -fPIC  -pedantic -Wall -Wno-long-long -Wwrite-strings    -DADD_ -fopenmp -lgfortran -O3 -DNDEBUG -DKOKKOS_DEPENDENCE -arch=sm_70 -shared -Wl,-soname,libkokk\
oskernels.so.15 -o libkokkoskernels.so.15.2.0 @CMakeFiles/kokkoskernels.dir/objects1.rsp  -Wl,-rpath,/scratch/gemmaops/jenkins/workspace/trilinos-ascicgpu-cde-cuda-openmp-shared-release/build/ascicgpu/trilinos_cde_v3-gnu-OpenMPI-cuda_11_2-OpenMP-shared-Release/packages/kokkos/simd\
/src:/scratch/gemmaops/jenkins/workspace/trilinos-ascicgpu-cde-cuda-openmp-shared-release/build/ascicgpu/trilinos_cde_v3-gnu-OpenMPI-cuda_11_2-OpenMP-shared-Release/packages/kokkos/algorithms/src:/scratch/gemmaops/jenkins/workspace/trilinos-ascicgpu-cde-cuda-openmp-shared-release/\
build/ascicgpu/trilinos_cde_v3-gnu-OpenMPI-cuda_11_2-OpenMP-shared-Release/packages/kokkos/containers/src:/scratch/gemmaops/jenkins/workspace/trilinos-ascicgpu-cde-cuda-openmp-shared-release/build/ascicgpu/trilinos_cde_v3-gnu-OpenMPI-cuda_11_2-OpenMP-shared-Release/packages/kokkos\
/core/src:/usr/lib/x86_64-redhat-linux6E/lib64: ../kokkos/simd/src/libkokkossimd.so.15.2.0 ../kokkos/algorithms/src/libkokkosalgorithms.so.15.2.0 ../kokkos/containers/src/libkokkoscontainers.so.15.2.0 ../kokkos/core/src/libkokkoscore.so.15.2.0 /projects/cde/v3/cee/spack/opt/spack/\
linux-rhel7-x86_64/gcc-10.3.0/cuda-11.2.2-5msjxkf5yufbjndzanbavtnyxslnxnkv/lib64/libcufft.so /projects/cde/v3/cee/spack/opt/spack/linux-rhel7-x86_64/gcc-10.3.0/cuda-11.2.2-5msjxkf5yufbjndzanbavtnyxslnxnkv/lib64/libcudart.so /usr/lib64/libcuda.so /usr/lib/x86_64-redhat-linux6E/lib6\
4/libdl.so /projects/cde/v3/cee/spack/opt/spack/linux-rhel7-x86_64/gcc-10.3.0/netlib-lapack-3.10.1-4eh2clciq6brd6y5upwusionwtlgq5eo/lib64/liblapack.so /projects/cde/v3/cee/spack/opt/spack/linux-rhel7-x86_64/gcc-10.3.0/netlib-lapack-3.10.1-4eh2clciq6brd6y5upwusionwtlgq5eo/lib64/lib\
blas.so /projects/cde/v3/cee/spack/opt/spack/linux-rhel7-x86_64/gcc-10.3.0/cuda-11.2.2-5msjxkf5yufbjndzanbavtnyxslnxnkv/lib64/libcublas.so /projects/cde/v3/cee/spack/opt/spack/linux-rhel7-x86_64/gcc-10.3.0/cuda-11.2.2-5msjxkf5yufbjndzanbavtnyxslnxnkv/lib64/libcusolver.so
CMakeFiles/kokkoskernels.dir/lapack/eti/generated_specializations_cpp/gesv/Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o: In function `KokkosLapack::Impl::CudaLapackSingleton::CudaLapackSingleton()':
tmpxft_00015892_00000000-6_Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cudafe1.cpp:(.text+0x490): multiple definition of `KokkosLapack::Impl::CudaLapackSingleton::CudaLapackSingleton()'
CMakeFiles/kokkoskernels.dir/lapack/tpls/KokkosLapack_Cuda_tpl.cpp.o:tmpxft_0001593d_00000000-6_KokkosLapack_Cuda_tpl.cudafe1.cpp:(.text+0x110): first defined here
CMakeFiles/kokkoskernels.dir/lapack/eti/generated_specializations_cpp/gesv/Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o: In function `KokkosLapack::Impl::CudaLapackSingleton::CudaLapackSingleton()':
tmpxft_00015892_00000000-6_Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cudafe1.cpp:(.text+0x490): multiple definition of `KokkosLapack::Impl::CudaLapackSingleton::CudaLapackSingleton()'
CMakeFiles/kokkoskernels.dir/lapack/tpls/KokkosLapack_Cuda_tpl.cpp.o:tmpxft_0001593d_00000000-6_KokkosLapack_Cuda_tpl.cudafe1.cpp:(.text+0x110): first defined here
CMakeFiles/kokkoskernels.dir/lapack/eti/generated_specializations_cpp/gesv/Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o: In function `KokkosLapack::Impl::CudaLapackSingleton::singleton()':
tmpxft_00015892_00000000-6_Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cudafe1.cpp:(.text+0x510): multiple definition of `KokkosLapack::Impl::CudaLapackSingleton::singleton()'
CMakeFiles/kokkoskernels.dir/lapack/tpls/KokkosLapack_Cuda_tpl.cpp.o:tmpxft_0001593d_00000000-6_KokkosLapack_Cuda_tpl.cudafe1.cpp:(.text+0x190): first defined here
CMakeFiles/kokkoskernels.dir/lapack/eti/generated_specializations_cpp/gesv/Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o: In function `KokkosLapack::Impl::MagmaSingleton::MagmaSingleton()':
tmpxft_00015892_00000000-6_Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cudafe1.cpp:(.text+0x580): multiple definition of `KokkosLapack::Impl::MagmaSingleton::MagmaSingleton()'
CMakeFiles/kokkoskernels.dir/lapack/tpls/KokkosLapack_Cuda_tpl.cpp.o:tmpxft_0001593d_00000000-6_KokkosLapack_Cuda_tpl.cudafe1.cpp:(.text+0x200): first defined here
CMakeFiles/kokkoskernels.dir/lapack/eti/generated_specializations_cpp/gesv/Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o: In function `KokkosLapack::Impl::MagmaSingleton::MagmaSingleton()':
tmpxft_00015892_00000000-6_Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cudafe1.cpp:(.text+0x580): multiple definition of `KokkosLapack::Impl::MagmaSingleton::MagmaSingleton()'
CMakeFiles/kokkoskernels.dir/lapack/tpls/KokkosLapack_Cuda_tpl.cpp.o:tmpxft_0001593d_00000000-6_KokkosLapack_Cuda_tpl.cudafe1.cpp:(.text+0x200): first defined here
CMakeFiles/kokkoskernels.dir/lapack/eti/generated_specializations_cpp/gesv/Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o: In function `KokkosLapack::Impl::MagmaSingleton::singleton()':
tmpxft_00015892_00000000-6_Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cudafe1.cpp:(.text+0x5f0): multiple definition of `KokkosLapack::Impl::MagmaSingleton::singleton()'
CMakeFiles/kokkoskernels.dir/lapack/tpls/KokkosLapack_Cuda_tpl.cpp.o:tmpxft_0001593d_00000000-6_KokkosLapack_Cuda_tpl.cudafe1.cpp:(.text+0x270): first defined here
CMakeFiles/kokkoskernels.dir/lapack/eti/generated_specializations_cpp/gesv/Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_SERIAL_MEMSPACE_HOSTSPACE.cpp.o: In function `KokkosLapack::Impl::CudaLapackSingleton::CudaLapackSingleton()':
tmpxft_0001597b_00000000-6_Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_SERIAL_MEMSPACE_HOSTSPACE.cudafe1.cpp:(.text+0x490): multiple definition of `KokkosLapack::Impl::CudaLapackSingleton::CudaLapackSingleton()'
CMakeFiles/kokkoskernels.dir/lapack/tpls/KokkosLapack_Cuda_tpl.cpp.o:tmpxft_0001593d_00000000-6_KokkosLapack_Cuda_tpl.cudafe1.cpp:(.text+0x110): first defined here
CMakeFiles/kokkoskernels.dir/lapack/eti/generated_specializations_cpp/gesv/Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_SERIAL_MEMSPACE_HOSTSPACE.cpp.o: In function `KokkosLapack::Impl::CudaLapackSingleton::CudaLapackSingleton()':
tmpxft_0001597b_00000000-6_Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_SERIAL_MEMSPACE_HOSTSPACE.cudafe1.cpp:(.text+0x490): multiple definition of `KokkosLapack::Impl::CudaLapackSingleton::CudaLapackSingleton()'
CMakeFiles/kokkoskernels.dir/lapack/tpls/KokkosLapack_Cuda_tpl.cpp.o:tmpxft_0001593d_00000000-6_KokkosLapack_Cuda_tpl.cudafe1.cpp:(.text+0x110): first defined here
CMakeFiles/kokkoskernels.dir/lapack/eti/generated_specializations_cpp/gesv/Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_SERIAL_MEMSPACE_HOSTSPACE.cpp.o: In function `KokkosLapack::Impl::CudaLapackSingleton::singleton()':
tmpxft_0001597b_00000000-6_Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_SERIAL_MEMSPACE_HOSTSPACE.cudafe1.cpp:(.text+0x510): multiple definition of `KokkosLapack::Impl::CudaLapackSingleton::singleton()'
CMakeFiles/kokkoskernels.dir/lapack/tpls/KokkosLapack_Cuda_tpl.cpp.o:tmpxft_0001593d_00000000-6_KokkosLapack_Cuda_tpl.cudafe1.cpp:(.text+0x190): first defined here
CMakeFiles/kokkoskernels.dir/lapack/eti/generated_specializations_cpp/gesv/Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_SERIAL_MEMSPACE_HOSTSPACE.cpp.o: In function `KokkosLapack::Impl::MagmaSingleton::MagmaSingleton()':
tmpxft_0001597b_00000000-6_Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_SERIAL_MEMSPACE_HOSTSPACE.cudafe1.cpp:(.text+0x580): multiple definition of `KokkosLapack::Impl::MagmaSingleton::MagmaSingleton()'
CMakeFiles/kokkoskernels.dir/lapack/tpls/KokkosLapack_Cuda_tpl.cpp.o:tmpxft_0001593d_00000000-6_KokkosLapack_Cuda_tpl.cudafe1.cpp:(.text+0x200): first defined here
CMakeFiles/kokkoskernels.dir/lapack/eti/generated_specializations_cpp/gesv/Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_SERIAL_MEMSPACE_HOSTSPACE.cpp.o: In function `KokkosLapack::Impl::MagmaSingleton::MagmaSingleton()':
tmpxft_0001597b_00000000-6_Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_SERIAL_MEMSPACE_HOSTSPACE.cudafe1.cpp:(.text+0x580): multiple definition of `KokkosLapack::Impl::MagmaSingleton::MagmaSingleton()'
CMakeFiles/kokkoskernels.dir/lapack/tpls/KokkosLapack_Cuda_tpl.cpp.o:tmpxft_0001593d_00000000-6_KokkosLapack_Cuda_tpl.cudafe1.cpp:(.text+0x200): first defined here
CMakeFiles/kokkoskernels.dir/lapack/eti/generated_specializations_cpp/gesv/Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_SERIAL_MEMSPACE_HOSTSPACE.cpp.o: In function `KokkosLapack::Impl::MagmaSingleton::singleton()':
tmpxft_0001597b_00000000-6_Lapack_gesv_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_SERIAL_MEMSPACE_HOSTSPACE.cudafe1.cpp:(.text+0x5f0): multiple definition of `KokkosLapack::Impl::MagmaSingleton::singleton()'
CMakeFiles/kokkoskernels.dir/lapack/tpls/KokkosLapack_Cuda_tpl.cpp.o:tmpxft_0001593d_00000000-6_KokkosLapack_Cuda_tpl.cudafe1.cpp:(.text+0x270): first defined here
CMakeFiles/kokkoskernels.dir/lapack/eti/generated_specializations_cpp/gesv/Lapack_gesv_eti_COMPLEX_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o: In function `KokkosLapack::Impl::CudaLapackSingleton::CudaLapackSingleton()':
tmpxft_00015a0f_00000000-6_Lapack_gesv_eti_COMPLEX_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cudafe1.cpp:(.text+0x490): multiple definition of `KokkosLapack::Impl::CudaLapackSingleton::CudaLapackSingleton()'
CMakeFiles/kokkoskernels.dir/lapack/tpls/KokkosLapack_Cuda_tpl.cpp.o:tmpxft_0001593d_00000000-6_KokkosLapack_Cuda_tpl.cudafe1.cpp:(.text+0x110): first defined here
CMakeFiles/kokkoskernels.dir/lapack/eti/generated_specializations_cpp/gesv/Lapack_gesv_eti_COMPLEX_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o: In function `KokkosLapack::Impl::CudaLapackSingleton::CudaLapackSingleton()':
tmpxft_00015a0f_00000000-6_Lapack_gesv_eti_COMPLEX_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cudafe1.cpp:(.text+0x490): multiple definition of `KokkosLapack::Impl::CudaLapackSingleton::CudaLapackSingleton()'
CMakeFiles/kokkoskernels.dir/lapack/tpls/KokkosLapack_Cuda_tpl.cpp.o:tmpxft_0001593d_00000000-6_KokkosLapack_Cuda_tpl.cudafe1.cpp:(.text+0x110): first defined here

glhenni avatar Apr 12 '24 16:04 glhenni

Pulling in @vqd8a who also works on the project

glhenni avatar Apr 12 '24 16:04 glhenni

I reproduced the error (or similar) by configuring with magma tpl enabled and cusparse disabled

ndellingwood avatar Apr 12 '24 19:04 ndellingwood

PR #2176 resolved the problem for me, once that goes through review and a final version is merged we'll patch to Trilinos

ndellingwood avatar Apr 12 '24 19:04 ndellingwood

For reference, this was the linker error (undefined ref) I was looking at:

../../libkokkoskernels.a(Sparse_gauss_seidel_apply_eti_DOUBLE_ORDINAL_INT_OFFSET_INT_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o): In function

`void KokkosSparse::spmv<Kokkos::Cuda, KokkosSparse::SPMVHandle<Kokkos::Cuda, KokkosSparse::CrsMatrix<double, int, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, void, int>, Kokkos::View<double**, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, Kokkos::MemoryTraits<1u> >, Kokkos::View<double**, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace> > >, double, KokkosSparse::CrsMatrix<double, int, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, void, int>, Kokkos::View<double**, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, Kokkos::MemoryTraits<1u> >, double, Kokkos::View<double**, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace> > >
(Kokkos::Cuda const&, KokkosSparse::SPMVHandle<Kokkos::Cuda, KokkosSparse::CrsMatrix<double, int, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, void, int>, Kokkos::View<double**, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, Kokkos::MemoryTraits<1u> >, Kokkos::View<double**, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace> > >*, char const*, double const&, KokkosSparse::CrsMatrix<double, int, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, void, int> const&, Kokkos::View<double**, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, Kokkos::MemoryTraits<1u> > const&, double const&, Kokkos::View<double**, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace> > const&)':

tmpxft_00375ef8_00000000-6_Sparse_gauss_seidel_apply_eti_DOUBLE_ORDINAL_INT_OFFSET_INT_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cudafe1.cpp:(.text._ZN12KokkosSparse4spmvIN6Kokkos4CudaENS_10SPMVHandleIS2_NS_9CrsMatrixIdiNS1_6DeviceIS2_NS1_9CudaSpaceEEEviEENS1_4ViewIPPdJNS1_10LayoutLeftES7_NS1_12MemoryTraitsILj1EEEEEENS9_ISB_JSC_S7_EEEEEdS8_SF_dSG_EEvRKT_PT0_PKcRKT1_RKT2_RKT3_RKT4_RKT5_[_ZN12KokkosSparse4spmvIN6Kokkos4CudaENS_10SPMVHandleIS2_NS_9CrsMatrixIdiNS1_6DeviceIS2_NS1_9CudaSpaceEEEviEENS1_4ViewIPPdJNS1_10LayoutLeftES7_NS1_12MemoryTraitsILj1EEEEEENS9_ISB_JSC_S7_EEEEEdS8_SF_dSG_EEvRKT_PT0_PKcRKT1_RKT2_RKT3_RKT4_RKT5_]+0x87c): undefined reference to
`KokkosSparse::Impl::SPMV_MV<Kokkos::Cuda, KokkosSparse::Impl::SPMVHandleImpl<Kokkos::Cuda, Kokkos::CudaSpace, double, int, int>, KokkosSparse::CrsMatrix<double const, int const, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, Kokkos::MemoryTraits<1u>, int const>, Kokkos::View<double const**, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, Kokkos::MemoryTraits<3u> >, Kokkos::View<double**, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, Kokkos::MemoryTraits<1u> >, false, true, true>::
spmv_mv(Kokkos::Cuda const&, KokkosSparse::Impl::SPMVHandleImpl<Kokkos::Cuda, Kokkos::CudaSpace, double, int, int>*, char const*, double const&, KokkosSparse::CrsMatrix<double const, int const, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, Kokkos::MemoryTraits<1u>, int const> const&, Kokkos::View<double const**, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, Kokkos::MemoryTraits<3u> > const&, double const&, Kokkos::View<double**, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, Kokkos::MemoryTraits<1u> > const&)'

what stood out was from the boolean template params to KokkosSparse::Impl::SPMV_MV, the second to last (i.e. tpl_spec_avail) is marked true, which should not be the case when magma is the only enabled TPL

ndellingwood avatar Apr 12 '24 19:04 ndellingwood

kokkos-kernels PR #2176 merged, applied the patch to Trilinos and submitted PR https://github.com/trilinos/Trilinos/pull/12916 with the fix

ndellingwood avatar Apr 12 '24 22:04 ndellingwood

We have magma available on weaver now and I added a nightly job testing magma, so we should catch these issues earlier going forward

ndellingwood avatar Apr 12 '24 22:04 ndellingwood

@glhenni the patch to Trilinos has merged https://github.com/trilinos/Trilinos/pull/12916 , hopefully your issues are resolved

ndellingwood avatar Apr 15 '24 17:04 ndellingwood

Thanks Nathan.

Keep in mind, I know almost nothing about kokkos and cuda. I'm just acting as the gemma team's devops guy. Given that, am I supposed to add a variable like cmake -DKOKKOSKERNELS_ENABLE_TPL_CUSPARSE=OFF to stop the compile error? Is it relevant only when KOKKOSKERNELS_ENABLE_TPL_MAGMA=ON? Are there performance implications with turning off CUSPARSE? Or do I just to -DTPL_ENABLE_CUSPARSE=OFF?

glhenni avatar Apr 16 '24 17:04 glhenni

Hi @glhenni , you shouldn't need to disable cusparse and hopefully your configuration is working now, but if not please post a cmake configuration of how you are building Trilinos as there may be other differences compared with what I tested. For this issue, when I tried to reproduce with magma I noticed the issue when cusparse was disabled (I just happened to be testing magma as the sole tpl), but both TPLs should work together.

ndellingwood avatar Apr 16 '24 18:04 ndellingwood

We use a lot of *.cmake files for settings, rather than putting everything on the command line, but I think the attached shell script should be representative of what's happening. You'll have to tweak the environment variables at the top for your specifics. cmake-only-tri.sh.txt and here is the output from that cmake cmake-out.txt and the error during "make" make-error-out.txt

glhenni avatar Apr 16 '24 19:04 glhenni

Thanks @glhenni , the files you shared were very helpful and I was able to reproduce (I had to also add -DCMAKE_EXE_LINKER_FLAGS="-L${MAGMA_ROOT}/lib -lmagma_sparse -lmagma" to get the libraries to link when trying to build tests).

There is an issue when magma and lapack are enabled simultaneously resulting in multiple definitions of their respective singletons; to resolve I split the definitions apart into separate headers and include in the appropriate cpp file, I was able to resolve the multiple definition issues, PR #2178

There are still some issues with magma+cublas in #2177 , which should be separate from the issues here

ndellingwood avatar Apr 17 '24 01:04 ndellingwood

The kokkos-kernels PR #2178 is merged; corresponding patch to Trilinos https://github.com/trilinos/Trilinos/pull/12938 has passed required testing, will merge shortly

ndellingwood avatar Apr 18 '24 22:04 ndellingwood

@ndellingwood Thanks for resolving this issue. @glhenni

vqd8a avatar Apr 19 '24 00:04 vqd8a