SyneRBI
docker: fix build
- [x] depends on #969
- [x] depends on #973
- [ ] revert debug commits
- unfortunately still needs
STIR_DISABLE_HDF5=ON - fixes #935
- closes #937
similar libcurl linker error:
#26 47.21 : && /usr/lib/ccache/g++ -DBOOST_ERROR_CODE_HEADER_ONLY -DBOOST_SYSTEM_NO_DEPRECATED -Wall -std=c++11 -Werror -O3 -DNDEBUG -rdynamic utilities/CMakeFiles/ismrmrd_test_xml.dir/ismrmrd_test_xml.cpp.o -o utilities/ismrmrd_test_xml -Wl,-rpath,/opt/SIRF-SuperBuild/builds/ISMRMRD/build:/usr/lib/x86_64-linux-gnu/hdf5/serial:/opt/conda/lib: libismrmrd.so.1.13.7 /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so /opt/conda/lib/libpugixml.so.1.15 && :
#26 47.21 /usr/bin/ld: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so: undefined reference to `curl_global_init@CURL_OPENSSL_4'
#26 47.21 /usr/bin/ld: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so: undefined reference to `curl_easy_perform@CURL_OPENSSL_4'
#26 47.21 /usr/bin/ld: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so: undefined reference to `curl_slist_free_all@CURL_OPENSSL_4'
#26 47.21 /usr/bin/ld: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so: undefined reference to `curl_easy_setopt@CURL_OPENSSL_4'
#26 47.21 /usr/bin/ld: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so: undefined reference to `curl_easy_init@CURL_OPENSSL_4'
#26 47.21 /usr/bin/ld: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so: undefined reference to `curl_slist_append@CURL_OPENSSL_4'
#26 47.21 /usr/bin/ld: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so: undefined reference to `curl_easy_cleanup@CURL_OPENSSL_4'
#26 47.21 /usr/bin/ld: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so: undefined reference to `curl_global_cleanup@CURL_OPENSSL_4'
#26 47.21 collect2: error: ld returned 1 exit status
#937 was along the way to do that (despite the title). However, it's obviously strange that our GHA work ok now, but docker doesn't, so probably some missing package somewhere.
The above linking error is caused by the system HDF5 not coming with hdf5-config.cmake. So CMake tries to get the information from h5cc. If that fails, it goes and finds some libraries, but has no clue what to link with, so really it should have given up already, but it valiantly forges ahead.
We normally really on the 2nd step, but we see https://github.com/SyneRBI/SIRF-SuperBuild/actions/runs/14171846317/job/39697234560#step:9:8335
#26 2.089 -- HDF5 C compiler wrapper is unable to compile a minimal HDF5 program.
#26 2.101 -- Found HDF5: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so (found suitable version "1.10.7", minimum required is "1.8")
#26 2.101 -- USING the system HDF5, found HDF5_INCLUDE_DIRS=/usr/include/hdf5/serial, HDF5_C_LIBRARY_hdf5=,HDF5_LIBRARIES=/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so
Sadly, CMake doesn't tell us why the HDF5 C compiler wrapper failed. It's stored in a log file though (can't remember which one). Maybe you could check if h5cc is really installed, although it should be https://github.com/SyneRBI/SIRF-SuperBuild/blob/b56b85bb4066fa6cffa854bbb0bcd5ed7804159d/docker/build_system-ubuntu.sh#L56-L59
Try and put
h5cc -showconfig
h5c++ -showconfig
before calling CMake. You can just as well stop the actual build for now while debugging.
looks like 'tis using /opt/conda/bin/h5c{c,++}
h5cc -showconfig
0.385 SUMMARY OF THE HDF5 CONFIGURATION
0.385 =================================
0.385
0.385 General Information:
0.385 -------------------
0.385 HDF5 Version: 1.14.3
0.385 Configured on: Wed Jan 22 03:13:54 UTC 2025
0.385 Configured by: conda@6f7718f776d2
0.385 Host system: x86_64-conda-linux-gnu
0.385 Uname information: Linux 6f7718f776d2 6.5.0-1025-azure #26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
0.385 Byte sex: little-endian
0.385 Installation point: /opt/conda
0.385
0.385 Compiling Options:
0.385 ------------------
0.385 Build Mode: production
0.385 Debugging Symbols: no
0.385 Asserts: no
0.385 Profiling: no
0.385 Optimization Level: high
0.385
0.385 Linking Options:
0.385 ----------------
0.385 Libraries: shared
0.385 Statically Linked Executables:
0.385 LDFLAGS: -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,/opt/conda/lib -Wl,-rpath-link,/opt/conda/lib -L/opt/conda/lib
0.385 H5_LDFLAGS:
0.385 AM_LDFLAGS: -L/opt/conda/lib -L/opt/conda/lib
0.385 Extra libraries: -lcrypto -lcurl -lpthread -lsz -lz -ldl -lm
0.385 Archiver: /home/conda/feedstock_root/build_artifacts/hdf5_1737515482579/_build_env/bin/x86_64-conda-linux-gnu-ar
0.385 AR_FLAGS: cr
0.385 Ranlib: /home/conda/feedstock_root/build_artifacts/hdf5_1737515482579/_build_env/bin/x86_64-conda-linux-gnu-ranlib
0.385
0.385 Languages:
0.385 ----------
0.385 C: yes
0.385 C Compiler: /home/conda/feedstock_root/build_artifacts/hdf5_1737515482579/_build_env/bin/x86_64-conda-linux-gnu-cc
0.385 CPPFLAGS: -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /opt/conda/include
0.385 H5_CPPFLAGS: -D_GNU_SOURCE -D_POSIX_C_SOURCE=200809L -DNDEBUG -UH5_DEBUG_API -I/home/conda/feedstock_root/build_artifacts/hdf5_1737515482579/work/src/H5FDsubfiling
0.385 AM_CPPFLAGS: -I/opt/conda/include -I/opt/conda/include
0.385 C Flags: -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /opt/conda/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/hdf5_1737515482579/work=/usr/local/src/conda/hdf5-1.14.3 -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix
0.385 H5 C Flags: -std=c99 -Wall -Wcast-qual -Wconversion -Wextra -Wfloat-equal -Wformat=2 -Winit-self -Winvalid-pch -Wmissing-include-dirs -Wshadow -Wundef -Wwrite-strings -pedantic -Wno-c++-compat -Wlarger-than=2560 -Wlogical-op -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wsync-nand -Wno-unsuffixed-float-constants -Wdouble-promotion -Wtrampolines -Wstack-usage=8192 -Wmaybe-uninitialized -Wdate-time -Warray-bounds=2 -Wc99-c11-compat -Wduplicated-cond -Whsa -Wnormalized -Wnull-dereference -Wunused-const-variable -Walloca -Walloc-zero -Wduplicated-branches -Wformat-overflow=2 -Wformat-truncation=1 -Wattribute-alias -Wcast-align=strict -Wshift-overflow=2 -Wattribute-alias=2 -Wmissing-profile -Wc11-c2x-compat -fstdarg-opt -fdiagnostics-urls=never -fno-diagnostics-color -s -Wbad-function-cast -Wcast-align -Wformat -Wimplicit-function-declaration -Wint-to-pointer-cast -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpacked -Wpointer-sign -Wpointer-to-int-cast -Wredundant-decls -Wstrict-prototypes -Wswitch -Wunused-but-set-variable -Wunused-variable -Wunused-function -Wunused-parameter -Wincompatible-pointer-types -Wint-conversion -Wshadow -Wrestrict -Wcast-function-type -Wmaybe-uninitialized -Wno-aggregate-return -Wno-inline -Wno-missing-format-attribute -Wno-missing-noreturn -Wno-overlength-strings -Wno-jump-misses-init -Wstrict-overflow=2 -Wno-suggest-attribute=const -Wno-suggest-attribute=noreturn -Wno-suggest-attribute=pure -Wno-suggest-attribute=format -Wno-suggest-attribute=cold -Wno-suggest-attribute=malloc -O3
0.385 AM C Flags:
0.385 Shared C Library: yes
0.385 Static C Library: no
0.385
0.385
0.385 Fortran: yes
0.385 Fortran Compiler: /home/conda/feedstock_root/build_artifacts/hdf5_1737515482579/_build_env/bin/x86_64-conda-linux-gnu-gfortran
0.385 Fortran Flags:
0.385 H5 Fortran Flags: -std=f2008 -Waliasing -Wall -Wcharacter-truncation -Wextra -Wimplicit-interface -Wsurprising -Wunderflow -pedantic -Wintrinsics-std -Wimplicit-procedure -Wreal-q-constant -Wfunction-elimination -Wrealloc-lhs -Wrealloc-lhs-all -Wno-c-binding-type -Winteger-division -Wfrontend-loop-interchange -fdiagnostics-urls=never -fno-diagnostics-color -s -Wno-unused-dummy-argument -Wno-array-temporaries -O3
0.385 AM Fortran Flags:
0.385 Shared Fortran Library: yes
0.385 Static Fortran Library: no
0.385 Module Directory: ${includedir}
0.385
0.385 C++: yes
0.385 C++ Compiler: /home/conda/feedstock_root/build_artifacts/hdf5_1737515482579/_build_env/bin/x86_64-conda-linux-gnu-c++
0.385 C++ Flags: -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /opt/conda/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/hdf5_1737515482579/work=/usr/local/src/conda/hdf5-1.14.3 -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix
0.385 H5 C++ Flags: -std=c++11 -Wall -Wcast-qual -Wconversion -Wctor-dtor-privacy -Weffc++ -Wextra -Wfloat-equal -Wformat=2 -Winit-self -Winvalid-pch -Wmissing-include-dirs -Wnon-virtual-dtor -Wold-style-cast -Woverloaded-virtual -Wreorder -Wshadow -Wsign-promo -Wundef -Wwrite-strings -pedantic -Wlarger-than=2560 -Wlogical-op -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wsync-nand -Wdouble-promotion -Wtrampolines -Wstack-usage=8192 -Wmaybe-uninitialized -Wdate-time -Wopenmp-simd -Warray-bounds=2 -Wduplicated-cond -Whsa -Wnormalized -Wnull-dereference -Wunused-const-variable -Walloca -Walloc-zero -Wduplicated-branches -Wformat-overflow=2 -Wformat-truncation=1 -Wattribute-alias -Wcast-align=strict -Wshift-overflow=2 -Wattribute-alias=2 -Wmissing-profile -Wno-deprecated-copy -fstdarg-opt -fdiagnostics-urls=never -fno-diagnostics-color -s -Wcast-align -Wmissing-declarations -Wpacked -Wredundant-decls -Wswitch -Wunused-but-set-variable -Wunused-function -Wunused-variable -Wunused-parameter -Wshadow -O3
0.385 AM C++ Flags:
0.385 Shared C++ Library: yes
0.385 Static C++ Library: no
0.385
0.385 Java: no
0.385
0.385
0.385 Features:
0.385 ---------
0.385 Parallel HDF5: no
0.385 Parallel Filtered Dataset Writes: no
0.385 Large Parallel I/O: no
0.385 High-level library: yes
0.385 Dimension scales w/ new references: no
0.385 Build HDF5 Tests: yes
0.385 Build HDF5 Tools: yes
0.385 Build GIF Tools: yes
0.385 Threadsafety: yes
0.385 Default API mapping: v114
0.385 With deprecated public symbols: yes
0.385 I/O filters (external): deflate(zlib),szip(encoder)
0.385 Map (H5M) API: no
0.385 Direct VFD: yes
0.385 Mirror VFD: no
0.385 Subfiling VFD: no
0.385 (Read-Only) S3 VFD: yes
0.385 (Read-Only) HDFS VFD: no
0.385 Packages w/ extra debug output: none
0.385 API tracing: no
0.385 Using memory checker: yes
0.385 Function stack tracing: no
0.385 Use file locking: best-effort
0.385 Strict file format checks: no
0.385 Optimization instrumentation: no
h5c++ -showconfig
0.389 SUMMARY OF THE HDF5 CONFIGURATION
0.389 =================================
0.389
0.389 General Information:
0.389 -------------------
0.389 HDF5 Version: 1.14.3
0.389 Configured on: Wed Jan 22 03:13:54 UTC 2025
0.389 Configured by: conda@6f7718f776d2
0.389 Host system: x86_64-conda-linux-gnu
0.389 Uname information: Linux 6f7718f776d2 6.5.0-1025-azure #26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
0.389 Byte sex: little-endian
0.389 Installation point: /opt/conda
0.389
0.389 Compiling Options:
0.389 ------------------
0.389 Build Mode: production
0.389 Debugging Symbols: no
0.389 Asserts: no
0.389 Profiling: no
0.389 Optimization Level: high
0.389
0.389 Linking Options:
0.389 ----------------
0.389 Libraries: shared
0.389 Statically Linked Executables:
0.389 LDFLAGS: -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,/opt/conda/lib -Wl,-rpath-link,/opt/conda/lib -L/opt/conda/lib
0.389 H5_LDFLAGS:
0.389 AM_LDFLAGS: -L/opt/conda/lib -L/opt/conda/lib
0.389 Extra libraries: -lcrypto -lcurl -lpthread -lsz -lz -ldl -lm
0.389 Archiver: /home/conda/feedstock_root/build_artifacts/hdf5_1737515482579/_build_env/bin/x86_64-conda-linux-gnu-ar
0.389 AR_FLAGS: cr
0.389 Ranlib: /home/conda/feedstock_root/build_artifacts/hdf5_1737515482579/_build_env/bin/x86_64-conda-linux-gnu-ranlib
0.389
0.389 Languages:
0.389 ----------
0.389 C: yes
0.389 C Compiler: /home/conda/feedstock_root/build_artifacts/hdf5_1737515482579/_build_env/bin/x86_64-conda-linux-gnu-cc
0.389 CPPFLAGS: -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /opt/conda/include
0.389 H5_CPPFLAGS: -D_GNU_SOURCE -D_POSIX_C_SOURCE=200809L -DNDEBUG -UH5_DEBUG_API -I/home/conda/feedstock_root/build_artifacts/hdf5_1737515482579/work/src/H5FDsubfiling
0.389 AM_CPPFLAGS: -I/opt/conda/include -I/opt/conda/include
0.389 C Flags: -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /opt/conda/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/hdf5_1737515482579/work=/usr/local/src/conda/hdf5-1.14.3 -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix
0.389 H5 C Flags: -std=c99 -Wall -Wcast-qual -Wconversion -Wextra -Wfloat-equal -Wformat=2 -Winit-self -Winvalid-pch -Wmissing-include-dirs -Wshadow -Wundef -Wwrite-strings -pedantic -Wno-c++-compat -Wlarger-than=2560 -Wlogical-op -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wsync-nand -Wno-unsuffixed-float-constants -Wdouble-promotion -Wtrampolines -Wstack-usage=8192 -Wmaybe-uninitialized -Wdate-time -Warray-bounds=2 -Wc99-c11-compat -Wduplicated-cond -Whsa -Wnormalized -Wnull-dereference -Wunused-const-variable -Walloca -Walloc-zero -Wduplicated-branches -Wformat-overflow=2 -Wformat-truncation=1 -Wattribute-alias -Wcast-align=strict -Wshift-overflow=2 -Wattribute-alias=2 -Wmissing-profile -Wc11-c2x-compat -fstdarg-opt -fdiagnostics-urls=never -fno-diagnostics-color -s -Wbad-function-cast -Wcast-align -Wformat -Wimplicit-function-declaration -Wint-to-pointer-cast -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpacked -Wpointer-sign -Wpointer-to-int-cast -Wredundant-decls -Wstrict-prototypes -Wswitch -Wunused-but-set-variable -Wunused-variable -Wunused-function -Wunused-parameter -Wincompatible-pointer-types -Wint-conversion -Wshadow -Wrestrict -Wcast-function-type -Wmaybe-uninitialized -Wno-aggregate-return -Wno-inline -Wno-missing-format-attribute -Wno-missing-noreturn -Wno-overlength-strings -Wno-jump-misses-init -Wstrict-overflow=2 -Wno-suggest-attribute=const -Wno-suggest-attribute=noreturn -Wno-suggest-attribute=pure -Wno-suggest-attribute=format -Wno-suggest-attribute=cold -Wno-suggest-attribute=malloc -O3
0.389 AM C Flags:
0.389 Shared C Library: yes
0.389 Static C Library: no
0.389
0.389
0.389 Fortran: yes
0.389 Fortran Compiler: /home/conda/feedstock_root/build_artifacts/hdf5_1737515482579/_build_env/bin/x86_64-conda-linux-gnu-gfortran
0.389 Fortran Flags:
0.389 H5 Fortran Flags: -std=f2008 -Waliasing -Wall -Wcharacter-truncation -Wextra -Wimplicit-interface -Wsurprising -Wunderflow -pedantic -Wintrinsics-std -Wimplicit-procedure -Wreal-q-constant -Wfunction-elimination -Wrealloc-lhs -Wrealloc-lhs-all -Wno-c-binding-type -Winteger-division -Wfrontend-loop-interchange -fdiagnostics-urls=never -fno-diagnostics-color -s -Wno-unused-dummy-argument -Wno-array-temporaries -O3
0.389 AM Fortran Flags:
0.389 Shared Fortran Library: yes
0.389 Static Fortran Library: no
0.389 Module Directory: ${includedir}
0.389
0.389 C++: yes
0.389 C++ Compiler: /home/conda/feedstock_root/build_artifacts/hdf5_1737515482579/_build_env/bin/x86_64-conda-linux-gnu-c++
0.389 C++ Flags: -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /opt/conda/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/hdf5_1737515482579/work=/usr/local/src/conda/hdf5-1.14.3 -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix
0.389 H5 C++ Flags: -std=c++11 -Wall -Wcast-qual -Wconversion -Wctor-dtor-privacy -Weffc++ -Wextra -Wfloat-equal -Wformat=2 -Winit-self -Winvalid-pch -Wmissing-include-dirs -Wnon-virtual-dtor -Wold-style-cast -Woverloaded-virtual -Wreorder -Wshadow -Wsign-promo -Wundef -Wwrite-strings -pedantic -Wlarger-than=2560 -Wlogical-op -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wsync-nand -Wdouble-promotion -Wtrampolines -Wstack-usage=8192 -Wmaybe-uninitialized -Wdate-time -Wopenmp-simd -Warray-bounds=2 -Wduplicated-cond -Whsa -Wnormalized -Wnull-dereference -Wunused-const-variable -Walloca -Walloc-zero -Wduplicated-branches -Wformat-overflow=2 -Wformat-truncation=1 -Wattribute-alias -Wcast-align=strict -Wshift-overflow=2 -Wattribute-alias=2 -Wmissing-profile -Wno-deprecated-copy -fstdarg-opt -fdiagnostics-urls=never -fno-diagnostics-color -s -Wcast-align -Wmissing-declarations -Wpacked -Wredundant-decls -Wswitch -Wunused-but-set-variable -Wunused-function -Wunused-variable -Wunused-parameter -Wshadow -O3
0.389 AM C++ Flags:
0.389 Shared C++ Library: yes
0.389 Static C++ Library: no
0.389
0.389 Java: no
0.389
0.389
0.389 Features:
0.389 ---------
0.389 Parallel HDF5: no
0.389 Parallel Filtered Dataset Writes: no
0.389 Large Parallel I/O: no
0.389 High-level library: yes
0.389 Dimension scales w/ new references: no
0.389 Build HDF5 Tests: yes
0.389 Build HDF5 Tools: yes
0.389 Build GIF Tools: yes
0.389 Threadsafety: yes
0.389 Default API mapping: v114
0.389 With deprecated public symbols: yes
0.389 I/O filters (external): deflate(zlib),szip(encoder)
0.389 Map (H5M) API: no
0.389 Direct VFD: yes
0.389 Mirror VFD: no
0.389 Subfiling VFD: no
0.389 (Read-Only) S3 VFD: yes
0.389 (Read-Only) HDFS VFD: no
0.389 Packages w/ extra debug output: none
0.389 API tracing: no
0.389 Using memory checker: yes
0.389 Function stack tracing: no
0.389 Use file locking: best-effort
0.389 Strict file format checks: no
0.389 Optimization instrumentation: no
while -DHDF5_ROOT=/usr/lib/x86_64-linux-gnu results in:
3.182 CMake Error at /usr/local/share/cmake-3.29/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
3.182 Could NOT find HDF5: (Required is at least version "1.8") (found
3.182 /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so, )
(I didn't know we could use html summary/details on GitHub. nice!)
It looks like h5cc is installed from conda hdf5, which is probably installed via h5py. However, somewhat crazily, the h5cc actually cannot compile anything, as it depends on c-compiler which we don't have currently. (IMHO, conda -hdf5 should therefore NOT include h5cc...). More info is in #935.
As I argued in #935: installing h5py means we now have 2 HDF5 versions installed (system and conda). I suppose we didn't have this before on docker, because we didn't install relevant CIL dependencies (or only installed it after building, although that's still dangerous, as anything in python would have a run-time of the conda-forge hdf5, while we would have compiled with system hdf5, which is a lot older, and I'm not so sure how good that'd work).
I guess I'd try the following:
- add
cxx-compilerandc-compilerto our .yml file, or have a "build .yml") - make sure that conda installs all of that before any cmake
- update hash for STIR in version_config.cmake to 4f73a6cf9b9ba78c4b0e259c3fd4d890b592077d (as we need https://github.com/UCL/STIR/pull/1580)
- activate env and run CMake (it should pick up conda-forge's c,c++ compilers and h5cc)
Our wiki page says however we might have a problem with CCPi-Regularisation-Toolkit, but I cannot remember what that was. Maybe @paskino knows.
If that doesn't work, we'll have to finish #937 or port bits from it (That PR got confused by trying Ubuntu 24.04, recent Gadgetron and all that, but I think we should avoid that for now).
sigh
Tried adding the c{,xx}-compiler & bumping STIR, and now the error is:
CMake Error at /usr/local/share/cmake-3.29/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find Boost (missing: system) (found suitable version "1.74.0",
minimum required is "1.71.0")
that's weird, as boost system is installed (and is found in the action log). can you push or send log?
I couldn't find the doc for this, but apparently, gcc 13.3 is too recent for boost 1.74.0. Solution: add boost to requirements.yml. And off we are on the slippery slope...
fails on (unrelated)
#15 14.15 + mamba list python
#15 14.15 + grep '^python '
#15 14.15 + tr -s ' '
#15 14.15 + cut -d ' ' -f 1,2
I have no idea why this happens (and have seen it before)
You reverted the submodule. I just force-pushed to remove the change.
You probably need to:
git checkout docker-bump
git fetch
git reset --hard origin/docker-bump
git submodule update
btw current error is now fftw3 related:
FFTW3 UNIX libraries: FFTW3_LIBRARY-NOTFOUND
CMake Error at /usr/local/share/cmake-3.29/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find FFTW3 (missing: FFTW3_LIBRARY)
Call Stack (most recent call first):
/usr/local/share/cmake-3.29/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
CMake/FindFFTW3.cmake:114 (find_package_handle_standard_args)
SuperBuild/External_FFTW3.cmake:96 (find_package)
CMake/ExternalProjectDependency.cmake:593 (include)
SuperBuild/External_ISMRMRD.cmake:31 (ExternalProject_Include_Dependencies)
CMake/ExternalProjectDependency.cmake:593 (include)
SuperBuild/External_SIRF.cmake:42 (ExternalProject_Include_Dependencies)
CMake/ExternalProjectDependency.cmake:593 (include)
CMake/ExternalProjectDependency.cmake:639 (ExternalProject_Include_Dependencies)
SuperBuild.cmake:325 (ExternalProject_Include_Dependencies)
CMakeLists.txt:42 (include)
Oops, sorry.
I haven't checked why this is the case, but the easiest will be to add fftw as well. Of course, we'll soon end up where @paskino was...
Adding fftw/fftw-static to conda (and even adding -DFFTW3_ROOT_DIR=${CONDA_DIR}) makes no difference to the error...
Adding
fftw/fftw-statictoconda(and even adding-DFFTW3_ROOT_DIR=${CONDA_DIR}) makes no difference to the error...
This makes very little sense to me, but I'm now trying to force conda to use g++-11 (otherwise we'll have Gadgetron failures anyway)
Pinning the C++ compiler apparently still needs boost package (i.e. it isn't happy with the apt one), and presumably all the others as well. Also, for whatever reason this gives us hdf4.
By the way, I see
#17 1.398 Warning: 'conda-forge' already in 'channels' list, moving to the top
#17 1.452 warning libmamba 'repo.anaconda.com', a commercial channel hosted by Anaconda.com, is used.
#17 1.452
#17 1.452 warning libmamba Please make sure you understand Anaconda Terms of Services.
Pinning the C++ compiler apparently still needs boost package (i.e. it isn't happy with the apt one), and presumably all the others as well.
I tried this in the VM. The APT libraries are in /usr/lib/x86_64-linux-gnu/. When using the system (i.e. APT) g++, CMake find the libraries there. When using the conda g++, CMake doesn't look there at all (it does look in /usr/lib). Presumably this is for a good reason...
So, we seem to be stuck here. if we need 'h5py` for CIL (@paskino @gfardell, please confirm), I see 2 options:
- go conda all the way (as started in #937), but pinning cxx-compiler
- stick to APT for now, remove
h5pyfromrequirements.ymland try pip (trying to math hdf5 version to the apt one)
I've brute-forced fixes for:
- the HDF5/cURL liker error on
External_ISMRMRD.cmake - the compiler standard version on
External_siemens_to_ismrmrd.cmake
There's now just this error:
ProjDataGEHDF5.cxx:(.text+0x847): undefined reference to
stir::GE::RDF_HDF5::GEHDF5Wrapper::read_sinogram(stir::Array<3, unsigned char>&, std::array<unsigned long, 3ul> const&, std::array<unsigned long, 3ul> const&)
presumably from UCL/STIR:src/buildblock/ProjDataGEHDF5.cxx:82
I'm not sure. It's vaguely similar to https://github.com/UCL/STIR/issues/1202. Could try to use a newer version of STIR. Possibly it's a STIR bug, although I have no clue why it would come up now.
However, if you have to patch ISMRMRD to link with HDF5, you'll have to STIR and SIRF as well.
As discussed at length in #935 and above: there's 2 versions of HDF5 libraries here: conda and apt. ISMRMRD [https://github.com/SyneRBI/SIRF-SuperBuild/actions/runs/15468133565/job/43545150850?pr=946#step:9:7945](got built with the APT version) as did STIR. However, CMake isn't happy with h5cc (Edit: I can't remember why see https://github.com/SyneRBI/SIRF-SuperBuild/issues/935#issuecomment-2651524936 above, and we still have h5py), and therefore doesn't know linking options for hdf5, hence the linking error.
(As stated in #935 as well: as the APT and conda HDF5 versions are so wildly different, I'd imagine this will cause run-time conflicts when running from conda's python (but maybe not).)
I'm not sure how to get out of this swamp TBH. In #937, we started on using conda for nearly everything on docker, but that also got complicated (one reason being that the VM and GHA don't use conda). Sigh.
Could try to use a newer version of STIR
I tried using DEVEL_BUILD i.e. STIR master, which still fails.
sorry, I only now see that this linking error is when building SIRF. STIR itself installed fine. It seems to be STIR's "circular library dependency problem", which seems only mentioned in one of our oldest issues https://github.com/UCL/STIR/issues/6.
But I have no clue why this appears here, and not in other SIRF builds (SIRF Actions are fine).
(As stated in #935 as well: as the APT and conda HDF5 versions are so wildly different, I'd imagine this will cause run-time conflicts when running from conda's python (but maybe not).)
also for curl now
Cannot generate a safe runtime search path for target ismrmrd because files
in some directories may conflict with libraries in implicit directories:
runtime library [libcurl.so.4] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/opt/conda/lib
and in other places libunwind.so.8, liblapacke.so.3, libopenblas.so.0, libgomp.so.1.
As I now spotted that we have still have conda h5py, can we just try to make sure that isn't installed (or force remove it before building), which will make the curl linking problem disappear, not sure about the STIR linking problem though.
If that seems to work, it might be safest to build our own HDF5 as well as install libhdf5 with conda, with matched versions, but we need to get through the build errors first.
regarding
There's now just this error:
ProjDataGEHDF5.cxx:(.text+0x847): undefined reference to stir::GE::RDF_HDF5::GEHDF5Wrapper::read_sinogram(stir::Array<3, unsigned char>&, std::array<unsigned long, 3ul> const&, std::array<unsigned long, 3ul> const&)presumably from UCL/STIR:src/buildblock/ProjDataGEHDF5.cxx:82
To diagnose if this is a STIR recursive library problem, let's try the following.
- execute the original line separately to see if we have the same error
- then execute the modified line by adding
libIO.aone more time, see below
cd /opt/SIRF-SuperBuild/builds/SIRF/build
/usr/lib/ccache/g++ -DBOOST_ERROR_CODE_HEADER_ONLY -DBOOST_SYSTEM_NO_DEPRECATED -O3 -DNDEBUG /opt/SIRF-SuperBuild/INSTALL/lib/objects-Release/stir_registries/buildblock/buildblock_registries.cxx.o /opt/SIRF-SuperBuild/INSTALL/lib/objects-Release/stir_registries/data_buildblock/data_buildblock_registries.cxx.o /opt/SIRF-SuperBuild/INSTALL/lib/objects-Release/stir_registries/IO/IO_registries.cxx.o /opt/SIRF-SuperBuild/INSTALL/lib/objects-Release/stir_registries/recon_buildblock/recon_buildblock_registries.cxx.o /opt/SIRF-SuperBuild/INSTALL/lib/objects-Release/stir_registries/Shape_buildblock/Shape_buildblock_registries.cxx.o /opt/SIRF-SuperBuild/INSTALL/lib/objects-Release/stir_registries/modelling_buildblock/modelling_registries.cxx.o /opt/SIRF-SuperBuild/INSTALL/lib/objects-Release/stir_registries/spatial_transformation_buildblock/spatial_transformation_registries.cxx.o /opt/SIRF-SuperBuild/INSTALL/lib/objects-Release/stir_registries/scatter_buildblock/scatter_registries.cxx.o src/Registration/cReg/CMakeFiles/sirf_flip_or_mirror_image.dir/sirf_flip_or_mirror_image.cpp.o -o src/Registration/cReg/sirf_flip_or_mirror_image -Wl,-rpath,/opt/SIRF-SuperBuild/INSTALL/lib:/opt/conda/lib:/usr/lib/x86_64-linux-gnu/hdf5/serial: src/Registration/cReg/libReg.a src/common/libcsirf.a src/Synergistic/cSyn/libcsyn.a src/xGadgetron/cGadgetron/libcgadgetron.a src/xSTIR/cSTIR/libcstir.a src/Registration/cReg/libReg.a src/common/libcsirf.a src/Synergistic/cSyn/libcsyn.a src/xGadgetron/cGadgetron/libcgadgetron.a src/xSTIR/cSTIR/libcstir.a src/Registration/NiftyMoMo/libNiftyMoMo.a /opt/SIRF-SuperBuild/INSTALL/lib/libreg_nifti.a /opt/SIRF-SuperBuild/INSTALL/lib/libz.a /opt/SIRF-SuperBuild/INSTALL/lib/libreg_png.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_ReadWriteImage.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_maths.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_tools.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_globalTrans.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_localTrans.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_measure.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_resampling.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_blockMatching.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_femTrans.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_aladin.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_f3d.a /opt/SIRF-SuperBuild/INSTALL/lib/libreg_nifti.a /opt/SIRF-SuperBuild/INSTALL/lib/libz.a /opt/SIRF-SuperBuild/INSTALL/lib/libreg_png.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_ReadWriteImage.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_maths.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_tools.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_globalTrans.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_localTrans.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_measure.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_resampling.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_blockMatching.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_femTrans.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_aladin.a /opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_f3d.a /opt/SIRF-SuperBuild/INSTALL/lib/libgadgetron_toolbox_cpunfft.so.4.5.0 /opt/SIRF-SuperBuild/INSTALL/lib/libgadgetron_toolbox_cpufft.so.4.5.0 /opt/SIRF-SuperBuild/INSTALL/lib/libgadgetron_toolbox_cpucore_math.so.4.5.0 -llapacke /usr/lib/x86_64-linux-gnu/liblapacke.so /usr/lib/x86_64-linux-gnu/libopenblas.so /usr/lib/x86_64-linux-gnu/libfftw3.so /usr/lib/x86_64-linux-gnu/libfftw3f.so /opt/SIRF-SuperBuild/INSTALL/lib/libgadgetron_toolbox_hostutils.so.4.5.0 /opt/SIRF-SuperBuild/INSTALL/lib/libgadgetron_core.so.4.5.0 /opt/SIRF-SuperBuild/INSTALL/lib/libgadgetron_toolbox_cpucore.so.4.5.0 /opt/SIRF-SuperBuild/INSTALL/lib/libismrmrd.so.1.13.7 /opt/conda/lib/libpugixml.so.1.15 /usr/lib/x86_64-linux-gnu/libcurl.so /opt/SIRF-SuperBuild/INSTALL/lib/libgadgetron_toolbox_log.so.4.5.0 src/iUtilities/libiutilities.a /usr/lib/x86_64-linux-gnu/libboost_system.so.1.74.0 /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.74.0 /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.74.0 /usr/lib/x86_64-linux-gnu/libboost_atomic.so.1.74.0 /usr/lib/x86_64-linux-gnu/libboost_date_time.so.1.74.0 /usr/lib/x86_64-linux-gnu/libboost_chrono.so.1.74.0 /opt/SIRF-SuperBuild/INSTALL/lib/libanalytic_FBP3DRP.a /opt/SIRF-SuperBuild/INSTALL/lib/libanalytic_FBP2D.a /opt/SIRF-SuperBuild/INSTALL/lib/libanalytic_SRT2D.a /opt/SIRF-SuperBuild/INSTALL/lib/libanalytic_SRT2DSPECT.a /opt/SIRF-SuperBuild/INSTALL/lib/libiterative_OSMAPOSL.a /opt/SIRF-SuperBuild/INSTALL/lib/libiterative_KOSMAPOSL.a /opt/SIRF-SuperBuild/INSTALL/lib/libiterative_OSSPS.a /opt/SIRF-SuperBuild/INSTALL/lib/libscatter_buildblock.a /opt/SIRF-SuperBuild/INSTALL/lib/librecon_buildblock.a /opt/SIRF-SuperBuild/INSTALL/lib/libparallelproj_c.so.1.9.0 /opt/SIRF-SuperBuild/INSTALL/lib/libdisplay.a /opt/SIRF-SuperBuild/INSTALL/lib/libspatial_transformation_buildblock.a /opt/SIRF-SuperBuild/INSTALL/lib/libShape_buildblock.a /opt/SIRF-SuperBuild/INSTALL/lib/libIO.a /opt/SIRF-SuperBuild/INSTALL/lib/libmodelling_buildblock.a /opt/SIRF-SuperBuild/INSTALL/lib/libIO.a /opt/SIRF-SuperBuild/INSTALL/lib/libmodelling_buildblock.a /opt/SIRF-SuperBuild/INSTALL/lib/liblistmode_buildblock.a /opt/SIRF-SuperBuild/INSTALL/lib/libdata_buildblock.a \
/opt/SIRF-SuperBuild/INSTALL/lib/libIO.a \
/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5_cpp.so /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so /opt/SIRF-SuperBuild/INSTALL/lib/libITKLabelMap-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKMathematicalMorphology-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKFastMarching-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKImageFeature-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKSmoothing-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKOptimizers-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKPolynomials-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKBiasCorrection-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKColormap-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKConvolution-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKStatistics-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libitkNetlibSlatec-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKSpatialObjects-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKPath-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKFFT-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKDenoising-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKDiffusionTensorImage-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIOJPEG2000-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libitkopenjpeg-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKQuadEdgeMeshFiltering-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKTestKernel-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIOBMP-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIOGDCM-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libitkgdcmMSFF-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libitkgdcmDICT-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libitkgdcmIOD-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libitkgdcmDSED-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libitkgdcmCommon-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIOGIPL-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIOJPEG-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIOMeshBYU-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIOMeshFreeSurfer-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIOMeshGifti-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKgiftiio-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKEXPAT-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIOMeshOBJ-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIOMeshOFF-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIOMeshVTK-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIOMeshBase-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKQuadEdgeMesh-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKMesh-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIOMeta-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKMetaIO-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIONIFTI-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKTransform-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKniftiio-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKznz-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIONRRD-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKNrrdIO-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIOPNG-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libitkpng-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIOTIFF-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libitktiff-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libitkzlib-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libitkjpeg-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIOVTK-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKIOImageBase-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKCommon-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libitkdouble-conversion-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libitksys-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libITKVNLInstantiation-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libitkvnl_algo-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libitkvnl-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libitkv3p_netlib-5.2.so.1 /opt/SIRF-SuperBuild/INSTALL/lib/libitkvcl-5.2.so.1 -lm -lm -ldl /opt/SIRF-SuperBuild/INSTALL/lib/libnumerics_buildblock.a /opt/SIRF-SuperBuild/INSTALL/lib/libeval_buildblock.a /opt/SIRF-SuperBuild/INSTALL/lib/libbuildblock.a /usr/lib/gcc/x86_64-linux-gnu/11/libgomp.so /usr/lib/x86_64-linux-gnu/libpthread.a -Wl,-rpath-link,/opt/SIRF-SuperBuild/INSTALL/lib
If this is a STIR bug, then we can set -DSTIR_DISABLE_HDF5=ON when building the docker image for now.