scr
scr copied to clipboard
LLNL
Static build fails with DTCMP/LWGRP error when examples enabled
Building SCR statically, with DTCMP/LWGRP also built statically, fails
First encountered this when testing the Spack builds. Was able to replicate it with bootstrap.sh.
How to Reproduce
Using bootstrap.sh from PR #452, build the deps.
Then configure scr with -DENABLE_IBM_BBAPI=OFF and -DBUILD_SHARED_LIBS=OFF and build:
$ ./boostrap.sh --debug --dev --static
$ mkdir -p build && cd build
$ cmake -DCMAKE_INSTALL_PREFIX=/path/to/scr/install -DSCR_RESOURCE_MANAGER=LSF -DENABLE_IBM_BBAPI=OFF -DBUILD_SHARED_LIBS=OFF ..
$ make VERBOSE=1
Resulting Error(s)
[ 87%] Building C object examples/CMakeFiles/test_api.dir/test_api.c.o
cd /usr/workspace/stanavig/trash/scr/build/examples && /usr/tcetmp/bin/cc -I/usr/workspace/stanavig/trash/scr/install/include -I/usr/tce/packages/spectrum-mpi/ibm/spectrum-mpi-2020.08.19/include -I/usr/workspace/stanavig/trash/scr/src -I/usr/workspace/stanavig/trash/scr/examples -I/usr/workspace/stanavig/trash/scr/build -Wall -fPIE -o CMakeFiles/test_api.dir/test_api.c.o -c /usr/workspace/stanavig/trash/scr/examples/test_api.c
[ 88%] Linking C executable test_api
cd /usr/workspace/stanavig/trash/scr/build/examples && /usr/tce/packages/cmake/cmake-3.14.5/bin/cmake -E cmake_link_script CMakeFiles/test_api.dir/link.txt --verbose=1
/usr/tcetmp/bin/cc -Wall -rdynamic CMakeFiles/test_api.dir/test_common.c.o CMakeFiles/test_api.dir/test_api.c.o -o test_api -Wl,-rpath,/usr/tce/packages/spectrum-mpi/ibm/spectrum-mpi-2020.08.19/lib ../src/libscr.a ../../install/lib64/libaxl.a ../../install/lib64/liber.a ../../install/lib64/libshuffile.a ../../install/lib64/libredset.a ../../install/lib64/libspath.a ../../install/lib64/librankstr.a ../../install/lib64/libkvtree.a ../../install/lib/libdtcmp.a /usr/tce/packages/spectrum-mpi/ibm/spectrum-mpi-2020.08.19/lib/libmpiprofilesupport.so /usr/tce/packages/spectrum-mpi/ibm/spectrum-mpi-2020.08.19/lib/libmpi_ibm.so -lyogrt -lz -lpthread
../../install/lib/libdtcmp.a(libdtcmp_la-dtcmp_sortv_cheng.o): In function `DTCMP_Sortv_cheng_lwgrp':
/usr/workspace/stanavig/trash/scr/deps/dtcmp-1.1.3/src/dtcmp_sortv_cheng.c:705: undefined reference to `lwgrp_comm_rank'
/usr/workspace/stanavig/trash/scr/deps/dtcmp-1.1.3/src/dtcmp_sortv_cheng.c:706: undefined reference to `lwgrp_comm_size'
../../install/lib/libdtcmp.a(libdtcmp_la-dtcmp_sortv_cheng.o): In function `find_exact_splitters':
/usr/workspace/stanavig/trash/scr/deps/dtcmp-1.1.3/src/dtcmp_sortv_cheng.c:196: undefined reference to `lwgrp_comm_size'
/usr/workspace/stanavig/trash/scr/deps/dtcmp-1.1.3/src/dtcmp_sortv_cheng.c:253: undefined reference to `lwgrp_comm_allgather'
../../install/lib/libdtcmp.a(libdtcmp_la-dtcmp_sortv_cheng.o): In function `compute_weighted_median':
/usr/workspace/stanavig/trash/scr/deps/dtcmp-1.1.3/src/dtcmp_sortv_cheng.c:82: undefined reference to `lwgrp_comm_size'
/usr/workspace/stanavig/trash/scr/deps/dtcmp-1.1.3/src/dtcmp_sortv_cheng.c:94: undefined reference to `lwgrp_comm_alltoall'
/usr/workspace/stanavig/trash/scr/deps/dtcmp-1.1.3/src/dtcmp_sortv_cheng.c:173: undefined reference to `lwgrp_comm_allgather'
../../install/lib/libdtcmp.a(libdtcmp_la-dtcmp_sortv_cheng.o): In function `find_exact_splitters':
/usr/workspace/stanavig/trash/scr/deps/dtcmp-1.1.3/src/dtcmp_sortv_cheng.c:353: undefined reference to `lwgrp_comm_allreduce'
../../install/lib/libdtcmp.a(libdtcmp_la-dtcmp_sortv_cheng.o): In function `compute_weighted_median':
/usr/workspace/stanavig/trash/scr/deps/dtcmp-1.1.3/src/dtcmp_sortv_cheng.c:173: undefined reference to `lwgrp_comm_allgather'
../../install/lib/libdtcmp.a(libdtcmp_la-dtcmp_sortv_cheng.o): In function `exact_split_exchange_merge':
/usr/workspace/stanavig/trash/scr/deps/dtcmp-1.1.3/src/dtcmp_sortv_cheng.c:530: undefined reference to `lwgrp_comm_size'
/usr/workspace/stanavig/trash/scr/deps/dtcmp-1.1.3/src/dtcmp_sortv_cheng.c:616: undefined reference to `lwgrp_comm_alltoall'
/usr/workspace/stanavig/trash/scr/deps/dtcmp-1.1.3/src/dtcmp_sortv_cheng.c:648: undefined reference to `lwgrp_comm_alltoallv'
../../install/lib/libdtcmp.a(libdtcmp_la-dtcmp_sortv_cheng.o): In function `find_exact_splitters':
/usr/workspace/stanavig/trash/scr/deps/dtcmp-1.1.3/src/dtcmp_sortv_cheng.c:422: undefined reference to `lwgrp_comm_alltoall'
../../install/lib/libdtcmp.a(libdtcmp_la-dtcmp_sortv_cheng.o): In function `DTCMP_Sortv_cheng':
/usr/workspace/stanavig/trash/scr/deps/dtcmp-1.1.3/src/dtcmp_sortv_cheng.c:806: undefined reference to `lwgrp_comm_build_from_mpicomm'
/usr/workspace/stanavig/trash/scr/deps/dtcmp-1.1.3/src/dtcmp_sortv_cheng.c:814: undefined reference to `lwgrp_comm_free'
collect2: error: ld returned 1 exit status
make[2]: *** [examples/CMakeFiles/test_api.dir/build.make:112: examples/test_api] Error 1
make[2]: Leaving directory '/usr/WS1/stanavig/trash/scr/build'
make[1]: *** [CMakeFiles/Makefile2:1770: examples/CMakeFiles/test_api.dir/all] Error 2
make[1]: Leaving directory '/usr/WS1/stanavig/trash/scr/build'
make: *** [Makefile:141: all] Error 2
Workaround
Disabling the examples allows the build to succeed:
$ cmake -DCMAKE_INSTALL_PREFIX=/path/to/scr/install -DSCR_RESOURCE_MANAGER=LSF -DBUILD_SHARED_LIBS=OFF -DENABLE_IBM_BBAPI=OFF -DENABLE_EXAMPLES=OFF ..
$ make
$ make install
