nczarr_tests fail for hdf5-1.12.0 for 4.9.0 release and main branch

[edwardhartnett]

For hdf5-1.12.0 nczarr_tests always fails for me:

PASS: run_chunkcases.sh
rm -f run_unknown.sh run_unknown.tmp
echo "#!/bin/bash" > run_unknown.tmp
echo "TESTNCZARR=1" >> run_unknown.tmp
cat ../nc_test4/tst_unknown.sh >> run_unknown.tmp
tr -d '\r' < run_unknown.tmp > run_unknown.sh
chmod a+x run_unknown.sh
rm -f run_unknown.tmp
PASS: run_nczarr_fill.sh
FAIL: run_nczfilter.sh
FAIL: run_filter.sh
FAIL: run_specific_filters.sh
FAIL: run_unknown.sh

For example, this happens with:

autoreconf -i && CPPFLAGS=-I/usr/local/hdf5-1.12.0/include LDFLAGS=-L/usr/local/hdf5-1.12.0/lib ./configure --disable-dap-remote-tests && make -j clean && make -j && make check This is not a parallel make issue, since it happens without the -j in make check.

Running one of the tests with bash -x:

bash -x ./run_quantize.sh
+ test x = x
++ pwd
+ srcdir=/home/ed/netcdf-c/nczarr_test
+ . ../test_common.sh
++ TOPSRCDIR=/home/ed/netcdf-c
++ TOPBUILDDIR=/home/ed/netcdf-c
++ FP_ISCMAKE=
++ FP_ISMSVC=
++ FP_WINVERMAJOR=0
++ FP_WINVERBUILD=0
++ FP_ISCYGWIN=
++ FP_ISMINGW=
++ FP_ISMSYS=
++ FP_ISOSX=
++ FP_ISREGEDIT=yes
++ FP_USEPLUGINS=yes
++ FEATURE_HDF5=yes
++ FEATURE_PARALLEL=no
++ FEATURE_HDF5=yes
++ FEATURE_NCZARR=yes
++ FEATURE_S3TESTS=no
++ FEATURE_NCZARR_ZIP=no
++ FEATURE_FILTERTESTS=yes
++ FEATURE_PLUGIN_INSTALL_DIR=/usr/local/hdf5/lib/plugin
++ set -e
++ test x = x1
+++ uname
++ system=Linux
++ test xLinux = x
++ top_srcdir=/home/ed/netcdf-c
++ top_builddir=/home/ed/netcdf-c
++ test x/home/ed/netcdf-c/nczarr_test = x
+++ pwd
++ builddir=/home/ed/netcdf-c/nczarr_test
++ execdir=/home/ed/netcdf-c/nczarr_test
+++ basename /home/ed/netcdf-c/nczarr_test
++ thisdir=nczarr_test
+++ pwd
++ WD=/home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
+++ pwd
++ srcdir=/home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c
+++ pwd
++ top_srcdir=/home/ed/netcdf-c
++ cd /home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
+++ pwd
++ builddir=/home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c
+++ pwd
++ top_builddir=/home/ed/netcdf-c
++ cd /home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
+++ pwd
++ execdir=/home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
++ export srcdir top_srcdir builddir top_builddir execdir
++ test -e /home/ed/netcdf-c/ncdump/ncdump.exe
++ ext=
++ export NCDUMP=/home/ed/netcdf-c/ncdump/ncdump
++ NCDUMP=/home/ed/netcdf-c/ncdump/ncdump
++ export NCCOPY=/home/ed/netcdf-c/ncdump/nccopy
++ NCCOPY=/home/ed/netcdf-c/ncdump/nccopy
++ export NCGEN=/home/ed/netcdf-c/ncgen/ncgen
++ NCGEN=/home/ed/netcdf-c/ncgen/ncgen
++ export NCGEN3=/home/ed/netcdf-c/ncgen3/ncgen3
++ NCGEN3=/home/ed/netcdf-c/ncgen3/ncgen3
++ export NCPATHCVT=/home/ed/netcdf-c/ncdump/ncpathcvt
++ NCPATHCVT=/home/ed/netcdf-c/ncdump/ncpathcvt
++ ncgen3c0=/home/ed/netcdf-c/ncgen3/c0.cdl
++ ncgenc0=/home/ed/netcdf-c/ncgen/c0.cdl
++ ncgenc04=/home/ed/netcdf-c/ncgen/c0_4.cdl
++ test x = xyes
++ test x = xyes
++ cd /home/ed/netcdf-c/nczarr_test
+ . /home/ed/netcdf-c/nczarr_test/test_nczarr.sh
++ test x '!=' x
++ test x = x
++ export NCZARR_S3_TEST_HOST=s3.us-east-1.amazonaws.com
++ NCZARR_S3_TEST_HOST=s3.us-east-1.amazonaws.com
++ test x = x
++ export NCZARR_S3_TEST_BUCKET=unidata-zarr-test-data
++ NCZARR_S3_TEST_BUCKET=unidata-zarr-test-data
++ export NCZARR_S3_TEST_URL=https://s3.us-east-1.amazonaws.com/unidata-zarr-test-data
++ NCZARR_S3_TEST_URL=https://s3.us-east-1.amazonaws.com/unidata-zarr-test-data
++ ZMD=/home/ed/netcdf-c/nczarr_test/zmapio
+++ pwd
++ WD=/home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
+++ pwd
++ abs_srcdir=/home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
+++ pwd
++ abs_execdir=/home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
+++ pwd
++ WD=/home/ed/netcdf-c/nczarr_test
++ test x '!=' x
++ test xyes = xyes
++ . /home/ed/netcdf-c/nczarr_test/findplugin.sh
+++ TOPSRCDIR=/home/ed/netcdf-c
+++ TOPBUILDDIR=/home/ed/netcdf-c
+++ test x/home/ed/netcdf-c/nczarr_test = x
+++ . /home/ed/netcdf-c/test_common.sh
++++ TOPSRCDIR=/home/ed/netcdf-c
++++ TOPBUILDDIR=/home/ed/netcdf-c
++++ FP_ISCMAKE=
++++ FP_ISMSVC=
++++ FP_WINVERMAJOR=0
++++ FP_WINVERBUILD=0
++++ FP_ISCYGWIN=
++++ FP_ISMINGW=
++++ FP_ISMSYS=
++++ FP_ISOSX=
++++ FP_ISREGEDIT=yes
++++ FP_USEPLUGINS=yes
++++ FEATURE_HDF5=yes
++++ FEATURE_PARALLEL=no
++++ FEATURE_HDF5=yes
++++ FEATURE_NCZARR=yes
++++ FEATURE_S3TESTS=no
++++ FEATURE_NCZARR_ZIP=no
++++ FEATURE_FILTERTESTS=yes
++++ FEATURE_PLUGIN_INSTALL_DIR=/usr/local/hdf5/lib/plugin
++++ set -e
++++ test x = x1
+++++ uname
++++ system=Linux
++++ test xLinux = x
++++ top_srcdir=/home/ed/netcdf-c
++++ top_builddir=/home/ed/netcdf-c
++++ test x/home/ed/netcdf-c/nczarr_test = x
+++++ pwd
++++ builddir=/home/ed/netcdf-c/nczarr_test
++++ execdir=/home/ed/netcdf-c/nczarr_test
+++++ basename /home/ed/netcdf-c/nczarr_test
++++ thisdir=nczarr_test
+++++ pwd
++++ WD=/home/ed/netcdf-c/nczarr_test
++++ cd /home/ed/netcdf-c/nczarr_test
+++++ pwd
++++ srcdir=/home/ed/netcdf-c/nczarr_test
++++ cd /home/ed/netcdf-c/nczarr_test
++++ cd /home/ed/netcdf-c
+++++ pwd
++++ top_srcdir=/home/ed/netcdf-c
++++ cd /home/ed/netcdf-c/nczarr_test
++++ cd /home/ed/netcdf-c/nczarr_test
+++++ pwd
++++ builddir=/home/ed/netcdf-c/nczarr_test
++++ cd /home/ed/netcdf-c/nczarr_test
++++ cd /home/ed/netcdf-c
+++++ pwd
++++ top_builddir=/home/ed/netcdf-c
++++ cd /home/ed/netcdf-c/nczarr_test
++++ cd /home/ed/netcdf-c/nczarr_test
+++++ pwd
++++ execdir=/home/ed/netcdf-c/nczarr_test
++++ cd /home/ed/netcdf-c/nczarr_test
++++ export srcdir top_srcdir builddir top_builddir execdir
++++ test -e /home/ed/netcdf-c/ncdump/ncdump.exe
++++ ext=
++++ export NCDUMP=/home/ed/netcdf-c/ncdump/ncdump
++++ NCDUMP=/home/ed/netcdf-c/ncdump/ncdump
++++ export NCCOPY=/home/ed/netcdf-c/ncdump/nccopy
++++ NCCOPY=/home/ed/netcdf-c/ncdump/nccopy
++++ export NCGEN=/home/ed/netcdf-c/ncgen/ncgen
++++ NCGEN=/home/ed/netcdf-c/ncgen/ncgen
++++ export NCGEN3=/home/ed/netcdf-c/ncgen3/ncgen3
++++ NCGEN3=/home/ed/netcdf-c/ncgen3/ncgen3
++++ export NCPATHCVT=/home/ed/netcdf-c/ncdump/ncpathcvt
++++ NCPATHCVT=/home/ed/netcdf-c/ncdump/ncpathcvt
++++ ncgen3c0=/home/ed/netcdf-c/ncgen3/c0.cdl
++++ ncgenc0=/home/ed/netcdf-c/ncgen/c0.cdl
++++ ncgenc04=/home/ed/netcdf-c/ncgen/c0_4.cdl
++++ test x = xyes
++++ test x = xyes
++++ cd /home/ed/netcdf-c/nczarr_test
+++ unset HDF5_PLUGIN_DIR
+++ findpluginext
++++ find /home/ed/netcdf-c/plugins -name '*misc.so'
+++ TSO='/home/ed/netcdf-c/plugins/.libs/lib__nch5misc.so
/home/ed/netcdf-c/plugins/.libs/lib__nczmisc.so'
++++ find /home/ed/netcdf-c/plugins -name '*misc.dylib'
+++ TDY=
++++ find /home/ed/netcdf-c/plugins -name 'cyg*misc.dll'
+++ TCYG=
++++ find /home/ed/netcdf-c/plugins -name '*misc.dll'
+++ TDLL=
+++ test 'x/home/ed/netcdf-c/plugins/.libs/lib__nch5misc.so
/home/ed/netcdf-c/plugins/.libs/lib__nczmisc.so' '!=' x
+++ FP_PLUGIN_EXT=so
+++ FP_PLUGIN_PRE=lib__nc
+++ findplugindir
+++ FP_PLUGIN_DIR=
++++ pwd
+++ CURWD=/home/ed/netcdf-c/nczarr_test
+++ cd /home/ed/netcdf-c/plugins
++++ pwd
+++ FP_PLUGINS=/home/ed/netcdf-c/plugins
+++ cd /home/ed/netcdf-c/nczarr_test
+++ test x '!=' x -a x '!=' x
+++ test -e /home/ed/netcdf-c/plugins/.libs
+++ FP_PLUGIN_DIR=/home/ed/netcdf-c/plugins/.libs
+++ test x/home/ed/netcdf-c/plugins/.libs = x
++++ /home/ed/netcdf-c/ncdump/ncpathcvt -F /home/ed/netcdf-c/plugins/.libs
+++ FP_PLUGIN_DIR=/home/ed/netcdf-c/plugins/.libs
+++ HDF5_PLUGIN_DIR=/home/ed/netcdf-c/plugins/.libs
++ echo 'findplugin.sh loaded'
findplugin.sh loaded
++ findplugin h5misc
++ FP_NAME=h5misc
++ FP_PLUGIN_LIB=
++ FP_PLUGIN_LIB=lib__nch5misc.so
++ test -f /home/ed/netcdf-c/plugins/.libs/lib__nch5misc.so
++ HDF5_PLUGIN_LIB=lib__nch5misc.so
++ HDF5_PLUGIN_DIR=/home/ed/netcdf-c/plugins/.libs
++ return 0
++ echo 'final HDF5_PLUGIN_DIR=/home/ed/netcdf-c/plugins/.libs'
final HDF5_PLUGIN_DIR=/home/ed/netcdf-c/plugins/.libs
++ export HDF5_PLUGIN_PATH=/home/ed/netcdf-c/plugins/.libs
++ HDF5_PLUGIN_PATH=/home/ed/netcdf-c/plugins/.libs
++ resetrc
++ test x = x1
++ rm -f /home/ed/netcdf-c/nczarr_test/.dodsrc /home/ed/netcdf-c/nczarr_test/.daprc /home/ed/netcdf-c/nczarr_test/.ncrc
++ unset NCRCENV_IGNORE
++ unset NCRCENV_RC
++ unset DAPRCFILE
+ set -e
+ testcase file
+ zext=file
+ fileargs tmp_quantize mode=,file
+ f=tmp_quantize
+ frag=mode=,file
+ test xmode=,file = x
+ case "$zext" in
+ file=tmp_quantize.file
+ fileurl=file://tmp_quantize.file#mode=,file
+ case "$zext" in
+ template=file:///home/ed/netcdf-c/nczarr_test/%s.zarr#mode=zarr,file
+ /home/ed/netcdf-c/nczarr_test/test_quantize file:///home/ed/netcdf-c/nczarr_test/%s.zarr#mode=zarr,file

*** Testing netcdf-4 variable quantization functions.
**** testing quantization setting and error conditions...ok.
**** testing quantization handling of non-floats...ok.
**** testing quantization of scalars...ok.
**** testing quantization of one value...ok.
**** testing more quantization values...ok.
**** testing quantization of one value with type conversion...ok.
**** testing more quantization values with type conversion...ok.
**** testing more quantization values with default fill values...ok.
**** testing more quantization values with custom fill values...ok.
*** Checking BitGroom values with type conversion between ints and floats...ok.
./run_quantize.sh: line 12: 1475143 Segmentation fault      (core dumped) ${execdir}/test_quantize "$template"
(base) ed@koko:~/netcdf-c/nczarr_test$ 

Could this be a bug in the hdf5-1.12.0 release? These tests work fine for 1.12.1 and 1.12.2. (Also 1.10.6 through 1.10.9).

If this is a HDF5 1.12.0 problem, we should detect that version in configure/CMake, and error out, telling the user to upgrade HDF5 versions. (As we do for versions 1.10.0 through 1.10.4, which have a bug).

@DennisHeimbigner this should be easily reproducible for you - it happens every time for me with 1.12.0.

[edwardhartnett]

OK, I put up a PR with the HDF5 version changed to 1.12.0 in the CI, and caught the error there: https://github.com/Unidata/netcdf-c/runs/6857408643?check_suite_focus=true

And on my machine it happens every time with 1.12.0.

[edwardhartnett]

I have rebuilt everything and tried again, without using make -j, and get errors.

make[3]: Entering directory '/home/ed/netcdf-c/nczarr_test'
PASS: run_ut_chunk.sh
PASS: run_ut_map.sh
PASS: run_ut_mapapi.sh
PASS: run_ut_misc.sh
PASS: run_ncgen4.sh
PASS: run_fillonlyz.sh
PASS: run_chunkcases.sh
FAIL: run_quantize.sh
PASS: run_purezarr.sh
PASS: run_interop.sh
PASS: run_misc.sh
PASS: run_nczarr_fill.sh
PASS: run_jsonconvention.sh
FAIL: run_nczfilter.sh
FAIL: run_filter.sh
FAIL: run_unknown.sh
FAIL: run_specific_filters.sh

Output of run_unknown.sh:

findplugin.sh loaded
final HDF5_PLUGIN_DIR=/home/ed/netcdf-c/plugins/.libs
findplugin.sh loaded
*** Testing access to filter info when filter implementation is not available for map file
./run_unknown.sh: line 63: 298152 Segmentation fault      (core dumped) ${NCGEN} -lb -4 -o $fileurl ${srcdir}/../nc_test4/unknown.cdl
FAIL run_unknown.sh (exit status: 139)

bash -x run_nczfilter.sh
+ test x = x
++ pwd
+ srcdir=/home/ed/netcdf-c/nczarr_test
+ . ../test_common.sh
++ TOPSRCDIR=/home/ed/netcdf-c
++ TOPBUILDDIR=/home/ed/netcdf-c
++ FP_ISCMAKE=
++ FP_ISMSVC=
++ FP_WINVERMAJOR=0
++ FP_WINVERBUILD=0
++ FP_ISCYGWIN=
++ FP_ISMINGW=
++ FP_ISMSYS=
++ FP_ISOSX=
++ FP_ISREGEDIT=yes
++ FP_USEPLUGINS=yes
++ FEATURE_HDF5=yes
++ FEATURE_PARALLEL=no
++ FEATURE_HDF5=yes
++ FEATURE_NCZARR=yes
++ FEATURE_S3TESTS=no
++ FEATURE_NCZARR_ZIP=no
++ FEATURE_FILTERTESTS=yes
++ FEATURE_PLUGIN_INSTALL_DIR=/usr/local/hdf5/lib/plugin
++ set -e
++ test x = x1
+++ uname
++ system=Linux
++ test xLinux = x
++ top_srcdir=/home/ed/netcdf-c
++ top_builddir=/home/ed/netcdf-c
++ test x/home/ed/netcdf-c/nczarr_test = x
+++ pwd
++ builddir=/home/ed/netcdf-c/nczarr_test
++ execdir=/home/ed/netcdf-c/nczarr_test
+++ basename /home/ed/netcdf-c/nczarr_test
++ thisdir=nczarr_test
+++ pwd
++ WD=/home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
+++ pwd
++ srcdir=/home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c
+++ pwd
++ top_srcdir=/home/ed/netcdf-c
++ cd /home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
+++ pwd
++ builddir=/home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c
+++ pwd
++ top_builddir=/home/ed/netcdf-c
++ cd /home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
+++ pwd
++ execdir=/home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
++ export srcdir top_srcdir builddir top_builddir execdir
++ test -e /home/ed/netcdf-c/ncdump/ncdump.exe
++ ext=
++ export NCDUMP=/home/ed/netcdf-c/ncdump/ncdump
++ NCDUMP=/home/ed/netcdf-c/ncdump/ncdump
++ export NCCOPY=/home/ed/netcdf-c/ncdump/nccopy
++ NCCOPY=/home/ed/netcdf-c/ncdump/nccopy
++ export NCGEN=/home/ed/netcdf-c/ncgen/ncgen
++ NCGEN=/home/ed/netcdf-c/ncgen/ncgen
++ export NCGEN3=/home/ed/netcdf-c/ncgen3/ncgen3
++ NCGEN3=/home/ed/netcdf-c/ncgen3/ncgen3
++ export NCPATHCVT=/home/ed/netcdf-c/ncdump/ncpathcvt
++ NCPATHCVT=/home/ed/netcdf-c/ncdump/ncpathcvt
++ ncgen3c0=/home/ed/netcdf-c/ncgen3/c0.cdl
++ ncgenc0=/home/ed/netcdf-c/ncgen/c0.cdl
++ ncgenc04=/home/ed/netcdf-c/ncgen/c0_4.cdl
++ test x = xyes
++ test x = xyes
++ cd /home/ed/netcdf-c/nczarr_test
+ . /home/ed/netcdf-c/nczarr_test/test_nczarr.sh
++ test x '!=' x
++ test x = x
++ export NCZARR_S3_TEST_HOST=s3.us-east-1.amazonaws.com
++ NCZARR_S3_TEST_HOST=s3.us-east-1.amazonaws.com
++ test x = x
++ export NCZARR_S3_TEST_BUCKET=unidata-zarr-test-data
++ NCZARR_S3_TEST_BUCKET=unidata-zarr-test-data
++ export NCZARR_S3_TEST_URL=https://s3.us-east-1.amazonaws.com/unidata-zarr-test-data
++ NCZARR_S3_TEST_URL=https://s3.us-east-1.amazonaws.com/unidata-zarr-test-data
++ ZMD=/home/ed/netcdf-c/nczarr_test/zmapio
+++ pwd
++ WD=/home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
+++ pwd
++ abs_srcdir=/home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
+++ pwd
++ abs_execdir=/home/ed/netcdf-c/nczarr_test
++ cd /home/ed/netcdf-c/nczarr_test
+++ pwd
++ WD=/home/ed/netcdf-c/nczarr_test
++ test x '!=' x
++ test xyes = xyes
++ . /home/ed/netcdf-c/nczarr_test/findplugin.sh
+++ TOPSRCDIR=/home/ed/netcdf-c
+++ TOPBUILDDIR=/home/ed/netcdf-c
+++ test x/home/ed/netcdf-c/nczarr_test = x
+++ . /home/ed/netcdf-c/test_common.sh
++++ TOPSRCDIR=/home/ed/netcdf-c
++++ TOPBUILDDIR=/home/ed/netcdf-c
++++ FP_ISCMAKE=
++++ FP_ISMSVC=
++++ FP_WINVERMAJOR=0
++++ FP_WINVERBUILD=0
++++ FP_ISCYGWIN=
++++ FP_ISMINGW=
++++ FP_ISMSYS=
++++ FP_ISOSX=
++++ FP_ISREGEDIT=yes
++++ FP_USEPLUGINS=yes
++++ FEATURE_HDF5=yes
++++ FEATURE_PARALLEL=no
++++ FEATURE_HDF5=yes
++++ FEATURE_NCZARR=yes
++++ FEATURE_S3TESTS=no
++++ FEATURE_NCZARR_ZIP=no
++++ FEATURE_FILTERTESTS=yes
++++ FEATURE_PLUGIN_INSTALL_DIR=/usr/local/hdf5/lib/plugin
++++ set -e
++++ test x = x1
+++++ uname
++++ system=Linux
++++ test xLinux = x
++++ top_srcdir=/home/ed/netcdf-c
++++ top_builddir=/home/ed/netcdf-c
++++ test x/home/ed/netcdf-c/nczarr_test = x
+++++ pwd
++++ builddir=/home/ed/netcdf-c/nczarr_test
++++ execdir=/home/ed/netcdf-c/nczarr_test
+++++ basename /home/ed/netcdf-c/nczarr_test
++++ thisdir=nczarr_test
+++++ pwd
++++ WD=/home/ed/netcdf-c/nczarr_test
++++ cd /home/ed/netcdf-c/nczarr_test
+++++ pwd
++++ srcdir=/home/ed/netcdf-c/nczarr_test
++++ cd /home/ed/netcdf-c/nczarr_test
++++ cd /home/ed/netcdf-c
+++++ pwd
++++ top_srcdir=/home/ed/netcdf-c
++++ cd /home/ed/netcdf-c/nczarr_test
++++ cd /home/ed/netcdf-c/nczarr_test
+++++ pwd
++++ builddir=/home/ed/netcdf-c/nczarr_test
++++ cd /home/ed/netcdf-c/nczarr_test
++++ cd /home/ed/netcdf-c
+++++ pwd
++++ top_builddir=/home/ed/netcdf-c
++++ cd /home/ed/netcdf-c/nczarr_test
++++ cd /home/ed/netcdf-c/nczarr_test
+++++ pwd
++++ execdir=/home/ed/netcdf-c/nczarr_test
++++ cd /home/ed/netcdf-c/nczarr_test
++++ export srcdir top_srcdir builddir top_builddir execdir
++++ test -e /home/ed/netcdf-c/ncdump/ncdump.exe
++++ ext=
++++ export NCDUMP=/home/ed/netcdf-c/ncdump/ncdump
++++ NCDUMP=/home/ed/netcdf-c/ncdump/ncdump
++++ export NCCOPY=/home/ed/netcdf-c/ncdump/nccopy
++++ NCCOPY=/home/ed/netcdf-c/ncdump/nccopy
++++ export NCGEN=/home/ed/netcdf-c/ncgen/ncgen
++++ NCGEN=/home/ed/netcdf-c/ncgen/ncgen
++++ export NCGEN3=/home/ed/netcdf-c/ncgen3/ncgen3
++++ NCGEN3=/home/ed/netcdf-c/ncgen3/ncgen3
++++ export NCPATHCVT=/home/ed/netcdf-c/ncdump/ncpathcvt
++++ NCPATHCVT=/home/ed/netcdf-c/ncdump/ncpathcvt
++++ ncgen3c0=/home/ed/netcdf-c/ncgen3/c0.cdl
++++ ncgenc0=/home/ed/netcdf-c/ncgen/c0.cdl
++++ ncgenc04=/home/ed/netcdf-c/ncgen/c0_4.cdl
++++ test x = xyes
++++ test x = xyes
++++ cd /home/ed/netcdf-c/nczarr_test
+++ unset HDF5_PLUGIN_DIR
+++ findpluginext
++++ find /home/ed/netcdf-c/plugins -name '*misc.so'
+++ TSO='/home/ed/netcdf-c/plugins/.libs/lib__nch5misc.so
/home/ed/netcdf-c/plugins/.libs/lib__nczmisc.so'
++++ find /home/ed/netcdf-c/plugins -name '*misc.dylib'
+++ TDY=
++++ find /home/ed/netcdf-c/plugins -name 'cyg*misc.dll'
+++ TCYG=
++++ find /home/ed/netcdf-c/plugins -name '*misc.dll'
+++ TDLL=
+++ test 'x/home/ed/netcdf-c/plugins/.libs/lib__nch5misc.so
/home/ed/netcdf-c/plugins/.libs/lib__nczmisc.so' '!=' x
+++ FP_PLUGIN_EXT=so
+++ FP_PLUGIN_PRE=lib__nc
+++ findplugindir
+++ FP_PLUGIN_DIR=
++++ pwd
+++ CURWD=/home/ed/netcdf-c/nczarr_test
+++ cd /home/ed/netcdf-c/plugins
++++ pwd
+++ FP_PLUGINS=/home/ed/netcdf-c/plugins
+++ cd /home/ed/netcdf-c/nczarr_test
+++ test x '!=' x -a x '!=' x
+++ test -e /home/ed/netcdf-c/plugins/.libs
+++ FP_PLUGIN_DIR=/home/ed/netcdf-c/plugins/.libs
+++ test x/home/ed/netcdf-c/plugins/.libs = x
++++ /home/ed/netcdf-c/ncdump/ncpathcvt -F /home/ed/netcdf-c/plugins/.libs
+++ FP_PLUGIN_DIR=/home/ed/netcdf-c/plugins/.libs
+++ HDF5_PLUGIN_DIR=/home/ed/netcdf-c/plugins/.libs
++ echo 'findplugin.sh loaded'
findplugin.sh loaded
++ findplugin h5misc
++ FP_NAME=h5misc
++ FP_PLUGIN_LIB=
++ FP_PLUGIN_LIB=lib__nch5misc.so
++ test -f /home/ed/netcdf-c/plugins/.libs/lib__nch5misc.so
++ HDF5_PLUGIN_LIB=lib__nch5misc.so
++ HDF5_PLUGIN_DIR=/home/ed/netcdf-c/plugins/.libs
++ return 0
++ echo 'final HDF5_PLUGIN_DIR=/home/ed/netcdf-c/plugins/.libs'
final HDF5_PLUGIN_DIR=/home/ed/netcdf-c/plugins/.libs
++ export HDF5_PLUGIN_PATH=/home/ed/netcdf-c/plugins/.libs
++ HDF5_PLUGIN_PATH=/home/ed/netcdf-c/plugins/.libs
++ resetrc
++ test x = x1
++ rm -f /home/ed/netcdf-c/nczarr_test/.dodsrc /home/ed/netcdf-c/nczarr_test/.daprc /home/ed/netcdf-c/nczarr_test/.ncrc
++ unset NCRCENV_IGNORE
++ unset NCRCENV_RC
++ unset DAPRCFILE
+ set -e
+ pwd
/home/ed/netcdf-c/nczarr_test
+ /home/ed/netcdf-c/nczarr_test/tst_nczfilter
run_nczfilter.sh: line 14: 298598 Segmentation fault      (core dumped) ${execdir}/tst_nczfilter
(base) ed@koko:~/netcdf-c/nczarr_test$ 

[edwardhartnett]

Address sanitizer finds a problem:

findplugin.sh loaded
final HDF5_PLUGIN_DIR=/home/ed/netcdf-c/plugins/.libs
*** Testing dynamic filters using API for map=file
=================================================================
==359868==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x619000002d50 at pc 0x7f0cc2c8a638 bp 0x7ffe66215610 sp 0x7ffe66214db8
READ of size 4 at 0x619000002d50 thread T0
    #0 0x7f0cc2c8a637 in MemcmpInterceptorCommon(void*, int (*)(void const*, void const*, unsigned long), void const*, void const*, unsigned long) ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:860
    #1 0x7f0cc2c8aba6 in __interceptor_memcmp ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:892
    #2 0x7f0cc2c8aba6 in __interceptor_memcmp ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:887
    #3 0x7f0cc23fedc2 in H5MM__is_our_block /home/ed/Downloads/hdf5-1.12.0/src/H5MM.c:141
    #4 0x7f0cc23feefd in H5MM_xfree /home/ed/Downloads/hdf5-1.12.0/src/H5MM.c:534
    #5 0x7f0cc21ee3dc in H5free_memory /home/ed/Downloads/hdf5-1.12.0/src/H5.c:990
    #6 0x7f0cbbe5ca0e in H5Z_filter_bzip2 (/home/ed/netcdf-c/plugins/.libs/lib__nch5bzip2.so.0+0x6a0e)
    #7 0x7f0cc2a3c5fd in NCZ_applyfilterchain (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x2dd5fd)
    #8 0x7f0cc29f5d0f in put_chunk (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x296d0f)
    #9 0x7f0cc29f4b6f in NCZ_flush_chunk_cache (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x295b6f)
    #10 0x7f0cc2a123eb in ncz_sync_var (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x2b33eb)
    #11 0x7f0cc2a0f623 in ncz_sync_grp.localalias (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x2b0623)
    #12 0x7f0cc2a0df95 in ncz_sync_file (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x2aef95)
    #13 0x7f0cc2a009dc in ncz_sync_netcdf4_file (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x2a19dc)
    #14 0x7f0cc29fffc4 in ncz_closeorabort (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x2a0fc4)
    #15 0x7f0cc29ffe5f in NCZ_close (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x2a0e5f)
    #16 0x7f0cc27ed416 in nc_close (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x8e416)
    #17 0x55ca42873320 in test_bzip2 (/home/ed/netcdf-c/nczarr_test/.libs/testfilter+0x3320)
    #18 0x55ca42873878 in main (/home/ed/netcdf-c/nczarr_test/.libs/testfilter+0x3878)
    #19 0x7f0cc1b25d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #20 0x7f0cc1b25e3f in __libc_start_main_impl ../csu/libc-start.c:392
    #21 0x55ca42872584 in _start (/home/ed/netcdf-c/nczarr_test/.libs/testfilter+0x2584)

0x619000002d50 is located 48 bytes to the left of 1024-byte region [0x619000002d80,0x619000003180)
allocated by thread T0 here:
    #0 0x7f0cc2caa867 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x7f0cc29f6766 in get_chunk (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x297766)
    #2 0x7f0cc29f3d8b in NCZ_read_cache_chunk (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x294d8b)
    #3 0x7f0cc2a353e3 in readfromcache (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x2d63e3)
    #4 0x7f0cc2a32240 in NCZ_transfer (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x2d3240)
    #5 0x7f0cc2a3194d in NCZ_transferslice (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x2d294d)
    #6 0x7f0cc2a2c714 in NCZ_put_vars (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x2cd714)
    #7 0x7f0cc2a2ac32 in NCZ_put_vara (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x2cbc32)
    #8 0x7f0cc27fd5fb in NC_put_vara (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x9e5fb)
    #9 0x7f0cc27fd89a in NC_put_var (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0x9e89a)
    #10 0x7f0cc2801224 in nc_put_var (/home/ed/netcdf-c/liblib/.libs/libnetcdf.so.19+0xa2224)
    #11 0x55ca42873307 in test_bzip2 (/home/ed/netcdf-c/nczarr_test/.libs/testfilter+0x3307)
    #12 0x55ca42873878 in main (/home/ed/netcdf-c/nczarr_test/.libs/testfilter+0x3878)
    #13 0x7f0cc1b25d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

SUMMARY: AddressSanitizer: heap-buffer-overflow ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:860 in MemcmpInterceptorCommon(void*, int (*)(void const*, void const*, unsigned long), void const*, void const*, unsigned long)
Shadow bytes around the buggy address:
  0x0c327fff8550: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c327fff8560: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c327fff8570: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c327fff8580: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c327fff8590: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c327fff85a0: fa fa fa fa fa fa fa fa fa fa[fa]fa fa fa fa fa
  0x0c327fff85b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c327fff85c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c327fff85d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c327fff85e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c327fff85f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==359868==ABORTING
FAIL run_filter.sh (exit status: 1)

[DennisHeimbigner]

What ./configure options did you use when building hdf5-1.12.0 ?

[edwardhartnett]

Just prefix. No other configure option

[DennisHeimbigner]

I ran on ubuntu 21 with:

1. HDF5 build:

./configure --prefix /usr/local --enable-hl --enable-shared --enable-static --with-zlib --with-szlib=/usr/local --enable-ros3-vfd

2. netcdf-c master build:

export CPPFLAGS=-I/usr/local/include export LDFLAGS=-L/usr/local/lib ./configure --disable-dap-remote-tests make clean all check

All tests passed. So what am I doing that is different from what you are doing?

[DennisHeimbigner]

So I reran with 1.12.0 built as you describe and it still works. Some questions:

1. Do have szip installed?
2. Does this fail as a github action?

[edwardhartnett]

No szip.

Failed in ci here https://github.com/Unidata/netcdf-c/runs/6857408643?check_suite_focus=true

[edwardhartnett]

OK, I do a complete rebuild of everything and the build fails with hdf5-1.12.0, but with no other version I've tested.

[edwardhartnett]

I think the answer here may simply be to accept that hdf5-1.12.0 has some memory bug that we hit and so we should make the user upgrade to 1.12.1 or 1.12.2...

[DennisHeimbigner]

OK, I do a complete rebuild of everything and the build fails with hdf5-1.12.0, but with no other version I've tested.

Sounds definitive to me.

[WardF]

Adding a note to the RELEASE_NOTES header; this conclusion sounds agreeable to me.

[DennisHeimbigner]

I have found that run_interop.sh fails when using distcheck. I a working on a fix now.