ADIOS 2.7.1 Build Issues (Conda-Forge)

[ax3l]

trafficstars

Updating the conda-forge recipes from 2.7.0 to 2.7.1 raises the following issues:

The Install.Make.CXX11 test errors on macOS:

make[1]: Entering directory '$SRC_DIR/build/Testing/install/make/CXX11'
$BUILD_PREFIX/bin/x86_64-apple-darwin13.4.0-clang++ -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -std=c++14 -fmessage-length=0 -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/adios2-2.7.1 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -isysroot /Applications/Xcode_12.3.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk -o main.o -c main_nompi.cxx $SRC_DIR/build/testing/install/install/include 
clang-11: warning: $SRC_DIR/build/testing/install/install/include: 'linker' input unused [-Wunused-command-line-argument]
main_nompi.cxx:6:10: fatal error: 'adios2.h' file not found
#include <adios2.h>
         ^~~~~~~~~~
1 error generated.
make[1]: *** [Makefile:26: main.o] Error 1
make[1]: Leaving directory '$SRC_DIR/build/Testing/install/make/CXX11'
CMake Error at $SRC_DIR/testing/install/run_make.cmake:37 (message):
  Result of test was 2, should be 0

On Linux, the adios_fortran_test errors with linker problems to dependencies:

...
$BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: $SRC_DIR/build/testing/install/install/lib/libadios2_core.so.2: undefined reference to `H5Sget_simple_extent_type'
$BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: $SRC_DIR/build/testing/install/install/lib/libadios2_core.so.2: undefined reference to `H5Aclose'
$BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: $SRC_DIR/build/testing/install/install/lib/libadios2_core.so.2: undefined reference to `blosc_set_compressor'
$BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: $SRC_DIR/build/testing/install/install/lib/libadios2_core.so.2: undefined reference to `zfp_stream_close'
$BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: $SRC_DIR/build/testing/install/install/lib/libadios2_core.so.2: undefined reference to `png_destroy_write_struct@PNG16_0'
$BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: $SRC_DIR/build/testing/install/install/lib/libadios2_core.so.2: undefined reference to `H5Tset_size'
$BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: $SRC_DIR/build/testing/install/install/lib/libadios2_core.so.2: undefined reference to `H5Acreate2'
$BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: $SRC_DIR/build/testing/install/install/lib/libadios2_core.so.2: undefined reference to `H5Gget_objname_by_idx'
$BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: $SRC_DIR/build/testing/install/install/lib/libadios2_core.so.2: undefined reference to `H5P_CLS_DATASET_CREATE_ID_g'
$BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: $SRC_DIR/build/testing/install/install/lib/libadios2_core.so.2: undefined reference to `zmq_connect'
$BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: $SRC_DIR/build/testing/install/install/lib/libadios2_core.so.2: undefined reference to `blosc_set_nthreads'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:23: adios_fortran_test] Error 1
make[1]: Leaving directory '$SRC_DIR/build/testing/install/make/Fortran'
CMake Error at $SRC_DIR/testing/install/run_make.cmake:37 (message):
  Result of test was 2, should be 0

and on Windows the CMake generator fails:

CMake Error at cmake/ADIOSFunctions.cmake:73 (add_test):
-- Generating done
  Error evaluating generator expression:

    $<TARGET_FILE_DIR:adios2>

  Target "adios2" is not an executable or library.
Call Stack (most recent call first):
  testing/adios2/bindings/python/CMakeLists.txt:15 (python_add_test)


CMake Error at cmake/ADIOSFunctions.cmake:73 (add_test):
  Error evaluating generator expression:

    $<TARGET_FILE_DIR:adios2>

  Target "adios2" is not an executable or library.
Call Stack (most recent call first):
  testing/adios2/bindings/python/CMakeLists.txt:17 (python_add_test)


CMake Warning:
  Manually-specified variables were not used by the project:

    ADIOS2_BUILD_TESTING


CMake Generate step failed.  Build files cannot be regenerated correctly.

[ax3l]

ping @chuckatkins

[ax3l]

ping @chuckatkins @pnorbert

[ax3l]

ping @chuckatkins @pnorbert is anyone working on these issues?

Several of us need this for post-processing of large runs, because 2.7.1 shipped this patch, among others: #2592

[ax3l]

My work-around is to disable test builds and install tests (-DADIOS2_RUN_INSTALL_TEST=OFF). Would ideally turn them on again, because they are used as smoke tests in package managers.

[vicentebolea]

@ax3l It seems that some variables are not being passing down to the install tests. In the particular case of macOS, the CMake variables such as ' https://github.com/conda-forge/adios2-feedstock/blob/2f5d76640398af1af5298b4cd300706679177e4d/recipe/cross-osx.cmake':

set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)

are CMAKE specifc and are not passed down to the makefile inside those INSTALL tests. I am unfamiliar with Conda-forge and how it calls cmake.

While this is resolved you can further enable more tests in all the builds other than windows by dropping -DADIOS2_RUN_INSTALL_TEST=OFF and in the ctest invocation use a flag such as --exclude-regex "install\.make\*"

[ax3l]

Hi @vicentebolea,

thanks for the hint! Those files are actually outdated and now used anymore. Let me remove them to demonstrate in https://github.com/conda-forge/adios2-feedstock/pull/30

[ax3l]

Fixed.

[ax3l]

Oh wait, I still use ADIOS2_RUN_INSTALL_TEST=OFF.

[ax3l]

Jup, still failing: https://github.com/conda-forge/adios2-feedstock/pull/50

[ax3l]

Still failing with ADIOS2 v2.8.3

[vicentebolea]

Conda package is now updated to 2.9: re-enabling tests at: https://github.com/conda-forge/adios2-feedstock/pull/75

[vicentebolea]

Since https://github.com/conda-forge/adios2-feedstock/pull/75 the conda package does not test the Install.Make.* tests. The rationale of this decision is described in the PR. Closing this since we do not have any long term intention to re-enable it.