netcdf-c
netcdf-c copied to clipboard
Unidata
Conda-forge test failures redux
Found this when building 4.9.1rc2. The relevant build log lines are:
2022-11-21T22:42:54.5509815Z Start 58: ncdump_tst_nccopy4
2022-11-21T22:42:54.5515223Z
2022-11-21T22:42:54.5521378Z 58: Test command: /usr/bin/bash "-c" "export srcdir=$SRC_DIR/ncdump;export TOPSRCDIR=$SRC_DIR;$SRC_DIR/build-shared/ncdump/tst_nccopy4.sh tst_nccopy4.sh "
2022-11-21T22:42:54.5527574Z 58: Working Directory: $SRC_DIR/build-shared/ncdump
2022-11-21T22:42:54.5533277Z 58: Test timeout computed to be: 1500
2022-11-21T22:42:54.5801585Z 58:
2022-11-21T22:42:54.5807946Z 58: *** Testing groups.
2022-11-21T22:42:54.5814936Z 58: *** creating nested group file tst_group_data.nc...ok.
2022-11-21T22:42:54.5821116Z 58: *** Tests successful!
2022-11-21T22:42:54.5961033Z 58:
2022-11-21T22:42:54.5961571Z 58: *** Testing enums.
2022-11-21T22:42:54.5962067Z 58: *** creating enum test file tst_enum_data.nc...ok.
2022-11-21T22:42:54.5962545Z 58: *** Tests successful!
2022-11-21T22:42:54.6100110Z 58:
2022-11-21T22:42:54.6100746Z 58: *** Testing compound types.
2022-11-21T22:42:54.6101274Z 58: *** creating compound test file tst_comp.nc...ok.
2022-11-21T22:42:54.6101750Z 58: *** Tests successful!
2022-11-21T22:42:54.6238944Z 58:
2022-11-21T22:42:54.6239360Z 58: *** Testing compound types some more.
2022-11-21T22:42:54.6239893Z 58: *** creating another compound test file tst_comp2.nc...ok.
2022-11-21T22:42:54.6240378Z 58: *** Tests successful!
2022-11-21T22:42:54.6251984Z 58:
2022-11-21T22:42:54.6259605Z 58: *** Testing netCDF-4 features of nccopy on ncdump/*.nc files
2022-11-21T22:42:54.6268757Z 58: *** Test nccopy tst_comp.nc copy_of_tst_comp.nc ...
2022-11-21T22:42:54.6669010Z 58: *** compare with copy_of_tst_comp.cdl
2022-11-21T22:42:54.6696766Z 58: *** Test nccopy tst_comp2.nc copy_of_tst_comp2.nc ...
2022-11-21T22:42:54.7220679Z 58: *** compare with copy_of_tst_comp2.cdl
2022-11-21T22:42:54.7251077Z 58: *** Test nccopy tst_enum_data.nc copy_of_tst_enum_data.nc ...
2022-11-21T22:42:54.7643154Z 58: *** compare with copy_of_tst_enum_data.cdl
2022-11-21T22:42:54.7671076Z 58: *** Test nccopy tst_fillbug.nc copy_of_tst_fillbug.nc ...
2022-11-21T22:42:54.8070545Z test 42
2022-11-21T22:42:54.8070955Z Start 42: ncdump_tst_netcdf4
2022-11-21T22:42:54.8071197Z
2022-11-21T22:42:54.8072694Z 42: Test command: /usr/bin/bash "-c" "export srcdir=$SRC_DIR/ncdump;export TOPSRCDIR=$SRC_DIR;$SRC_DIR/build-shared/ncdump/tst_netcdf4.sh tst_netcdf4.sh "
2022-11-21T22:42:54.8073642Z 42: Working Directory: $SRC_DIR/build-shared/ncdump
2022-11-21T22:42:54.8074331Z 42: Test timeout computed to be: 1500
2022-11-21T22:42:54.8089713Z 58: *** compare with copy_of_tst_fillbug.cdl
2022-11-21T22:42:54.8120318Z 58: *** Test nccopy tst_group_data.nc copy_of_tst_group_data.nc ...
2022-11-21T22:42:54.8203182Z 42:
2022-11-21T22:42:54.8204441Z 42: *** Testing ncgen and ncdump for netCDF-4 format.
2022-11-21T22:42:54.8573796Z 58: *** compare with copy_of_tst_group_data.cdl
2022-11-21T22:42:54.8616944Z 58: *** Test nccopy tst_nans.nc copy_of_tst_nans.nc ...
2022-11-21T22:42:54.8953182Z 58: *** compare with copy_of_tst_nans.cdl
2022-11-21T22:42:54.8975355Z 42: *** Creating test output tst_netcdf4_c0.nc.
2022-11-21T22:42:54.8990426Z 58: *** Test nccopy tst_opaque_data.nc copy_of_tst_opaque_data.nc ...
2022-11-21T22:42:54.9342465Z 42: *** Testing that program tst_h_rdc0 can read tst_netcdf4_c0.nc.
2022-11-21T22:42:54.9396887Z 58: *** compare with copy_of_tst_opaque_data.cdl
2022-11-21T22:42:54.9426702Z 58: *** Test nccopy tst_solar_1.nc copy_of_tst_solar_1.nc ...
2022-11-21T22:42:54.9487930Z 42:
2022-11-21T22:42:54.9488529Z 42: *** Checking HDF5 file c0.nc.
2022-11-21T22:42:54.9489005Z 42: *** Checking HDF5 objcts...ok.
2022-11-21T22:42:54.9493342Z 42: *** Tests successful!
2022-11-21T22:42:54.9493902Z 42: *** Running tst_create_files.c to create test files.
2022-11-21T22:42:54.9585756Z 42:
2022-11-21T22:42:54.9586277Z 42: *** Creating test files for ncdump.
2022-11-21T22:42:54.9586932Z 42: *** creating nested group file tst_solar_1.nc...Sorry! Unexpected result, $SRC_DIR/ncdump/tst_create_files.c, line: 71
2022-11-21T22:42:54.9607207Z 49/229 Test #42: ncdump_tst_netcdf4 ....................***Failed 0.15 sec
2022-11-21T22:42:54.9607700Z
2022-11-21T22:42:54.9608794Z *** Testing ncgen and ncdump for netCDF-4 format.
2022-11-21T22:42:54.9609342Z *** Creating test output tst_netcdf4_c0.nc.
2022-11-21T22:42:54.9609907Z *** Testing that program tst_h_rdc0 can read tst_netcdf4_c0.nc.
2022-11-21T22:42:54.9610229Z
2022-11-21T22:42:54.9610601Z *** Checking HDF5 file c0.nc.
2022-11-21T22:42:54.9611028Z *** Checking HDF5 objcts...ok.
2022-11-21T22:42:54.9611448Z *** Tests successful!
2022-11-21T22:42:54.9611900Z *** Running tst_create_files.c to create test files.
2022-11-21T22:42:54.9612197Z
2022-11-21T22:42:54.9612567Z *** Creating test files for ncdump.
2022-11-21T22:42:54.9614238Z *** creating nested group file tst_solar_1.nc...Sorry! Unexpected result, $SRC_DIR/ncdump/tst_create_files.c, line: 71
2022-11-21T22:42:54.9719951Z 58: $SRC_DIR/build-shared/ncdump/ncdump: tst_solar_1.nc: NetCDF: Unknown file format
2022-11-21T22:42:54.9748238Z 50/229 Test #58: ncdump_tst_nccopy4 ....................***Failed 0.42 sec
Full log is available if needed; here I snipped out unrelated tests.
Again, the fact that the tests failing are running concurrently is suspicious, so poke around and find that both ncdump/tst_netcdf4.sh and ncdump/tst_nccopy4.sh reference tst_solar_1.nc (and cdl) as well as tst_solar_2.nc (and cdl).
I also see mention of tst_solar across the codebase:
ncdump/tst_create_files.c
16:#define FILE_NAME_1 "tst_solar_1.nc"
17:#define FILE_NAME_2 "tst_solar_2.nc"
18:#define FILE_NAME_CMP "tst_solar_cmp.nc"
ncdump/tst_nccopy4.sh
23: tst_group_data tst_nans tst_opaque_data tst_solar_1 tst_solar_2
24: tst_solar_cmp tst_special_atts'
ncdump/tst_ncgen_shared.sh
90:ref_tst_solar_1 \
91:ref_tst_solar_2 \
140:BIGTESTS4="ref_tst_solar_1"
ncdump/tst_netcdf4.sh
47:${NCDUMP} tst_solar_1.nc | sed 's/e+0/e+/g' > tst_solar_1.cdl ; ERR
48:diff -b tst_solar_1.cdl $srcdir/ref_tst_solar_1.cdl ; ERR
49:${NCDUMP} tst_solar_2.nc | sed 's/e+0/e+/g' > tst_solar_2.cdl ; ERR
50:diff -b tst_solar_2.cdl $srcdir/ref_tst_solar_2.cdl ; ERR
ncgen/ctests.sh
84:ref_tst_solar_1 \
85:ref_tst_solar_2 \
104:BIGTESTS4="ref_tst_solar_1"
ncgen/ftests.sh
85:ref_tst_solar_1 \
86:ref_tst_solar_2 \
105:BIGTESTS4="ref_tst_solar_1"
ncgen/jtests.sh
84:ref_tst_solar_1 \
85:ref_tst_solar_2 \
108:BIGTESTS4="ref_tst_solar_1"
nczarr_test/run_ncgen4.sh
Is there a better way to approach this testing than playing "whack a mole" by manually renaming files? This is proving pretty brittle.
Unfortunately, many of our early tests in nc_test, nc_test4, and ncdump especially use files supposedly created in tests presumed to have run previously. Doing parallel make tests has completely screwed this up. The only practical solution I can figure are:
- give up on parallel testing or
- build a special global test script for these directories that guarantees serial execution.
- ?
=Dennis
In 11/21/2022 4:06 PM, Ryan May wrote:
Found this when building 4.9.1rc2 https://github.com/conda-forge/libnetcdf-feedstock/pull/154. The relevant build log lines are:
|2022-11-21T22:42:54.5509815Z Start 58: ncdump_tst_nccopy4 2022-11-21T22:42:54.5515223Z 2022-11-21T22:42:54.5521378Z 58: Test command: /usr/bin/bash "-c" "export srcdir=$SRC_DIR/ncdump;export TOPSRCDIR=$SRC_DIR;$SRC_DIR/build-shared/ncdump/tst_nccopy4.sh tst_nccopy4.sh " 2022-11-21T22:42:54.5527574Z 58: Working Directory: $SRC_DIR/build-shared/ncdump 2022-11-21T22:42:54.5533277Z 58: Test timeout computed to be: 1500 2022-11-21T22:42:54.5801585Z 58: 2022-11-21T22:42:54.5807946Z 58: *** Testing groups. 2022-11-21T22:42:54.5814936Z 58: *** creating nested group file tst_group_data.nc...ok. 2022-11-21T22:42:54.5821116Z 58: *** Tests successful! 2022-11-21T22:42:54.5961033Z 58: 2022-11-21T22:42:54.5961571Z 58: *** Testing enums. 2022-11-21T22:42:54.5962067Z 58: *** creating enum test file tst_enum_data.nc...ok. 2022-11-21T22:42:54.5962545Z 58: *** Tests successful! 2022-11-21T22:42:54.6100110Z 58: 2022-11-21T22:42:54.6100746Z 58: *** Testing compound types. 2022-11-21T22:42:54.6101274Z 58: *** creating compound test file tst_comp.nc...ok. 2022-11-21T22:42:54.6101750Z 58: *** Tests successful! 2022-11-21T22:42:54.6238944Z 58: 2022-11-21T22:42:54.6239360Z 58: *** Testing compound types some more. 2022-11-21T22:42:54.6239893Z 58: *** creating another compound test file tst_comp2.nc...ok. 2022-11-21T22:42:54.6240378Z 58: *** Tests successful! 2022-11-21T22:42:54.6251984Z 58: 2022-11-21T22:42:54.6259605Z 58: *** Testing netCDF-4 features of nccopy on ncdump/*.nc files 2022-11-21T22:42:54.6268757Z 58: *** Test nccopy tst_comp.nc copy_of_tst_comp.nc ... 2022-11-21T22:42:54.6669010Z 58: *** compare with copy_of_tst_comp.cdl 2022-11-21T22:42:54.6696766Z 58: *** Test nccopy tst_comp2.nc copy_of_tst_comp2.nc ... 2022-11-21T22:42:54.7220679Z 58: *** compare with copy_of_tst_comp2.cdl 2022-11-21T22:42:54.7251077Z 58: *** Test nccopy tst_enum_data.nc copy_of_tst_enum_data.nc ... 2022-11-21T22:42:54.7643154Z 58: *** compare with copy_of_tst_enum_data.cdl 2022-11-21T22:42:54.7671076Z 58: *** Test nccopy tst_fillbug.nc copy_of_tst_fillbug.nc ... 2022-11-21T22:42:54.8070545Z test 42 2022-11-21T22:42:54.8070955Z Start 42: ncdump_tst_netcdf4 2022-11-21T22:42:54.8071197Z 2022-11-21T22:42:54.8072694Z 42: Test command: /usr/bin/bash "-c" "export srcdir=$SRC_DIR/ncdump;export TOPSRCDIR=$SRC_DIR;$SRC_DIR/build-shared/ncdump/tst_netcdf4.sh tst_netcdf4.sh " 2022-11-21T22:42:54.8073642Z 42: Working Directory: $SRC_DIR/build-shared/ncdump 2022-11-21T22:42:54.8074331Z 42: Test timeout computed to be: 1500 2022-11-21T22:42:54.8089713Z 58: *** compare with copy_of_tst_fillbug.cdl 2022-11-21T22:42:54.8120318Z 58: *** Test nccopy tst_group_data.nc copy_of_tst_group_data.nc ... 2022-11-21T22:42:54.8203182Z 42: 2022-11-21T22:42:54.8204441Z 42: *** Testing ncgen and ncdump for netCDF-4 format. 2022-11-21T22:42:54.8573796Z 58: *** compare with copy_of_tst_group_data.cdl 2022-11-21T22:42:54.8616944Z 58: *** Test nccopy tst_nans.nc copy_of_tst_nans.nc ... 2022-11-21T22:42:54.8953182Z 58: *** compare with copy_of_tst_nans.cdl 2022-11-21T22:42:54.8975355Z 42: *** Creating test output tst_netcdf4_c0.nc. 2022-11-21T22:42:54.8990426Z 58: *** Test nccopy tst_opaque_data.nc copy_of_tst_opaque_data.nc ... 2022-11-21T22:42:54.9342465Z 42: *** Testing that program tst_h_rdc0 can read tst_netcdf4_c0.nc. 2022-11-21T22:42:54.9396887Z 58: *** compare with copy_of_tst_opaque_data.cdl 2022-11-21T22:42:54.9426702Z 58: *** Test nccopy tst_solar_1.nc copy_of_tst_solar_1.nc ... 2022-11-21T22:42:54.9487930Z 42: 2022-11-21T22:42:54.9488529Z 42: *** Checking HDF5 file c0.nc. 2022-11-21T22:42:54.9489005Z 42: *** Checking HDF5 objcts...ok. 2022-11-21T22:42:54.9493342Z 42: *** Tests successful! 2022-11-21T22:42:54.9493902Z 42: *** Running tst_create_files.c to create test files. 2022-11-21T22:42:54.9585756Z 42: 2022-11-21T22:42:54.9586277Z 42: *** Creating test files for ncdump. 2022-11-21T22:42:54.9586932Z 42: *** creating nested group file tst_solar_1.nc...Sorry! Unexpected result, $SRC_DIR/ncdump/tst_create_files.c, line: 71 2022-11-21T22:42:54.9607207Z 49/229 Test #42: ncdump_tst_netcdf4 ....................***Failed 0.15 sec 2022-11-21T22:42:54.9607700Z 2022-11-21T22:42:54.9608794Z *** Testing ncgen and ncdump for netCDF-4 format. 2022-11-21T22:42:54.9609342Z *** Creating test output tst_netcdf4_c0.nc. 2022-11-21T22:42:54.9609907Z *** Testing that program tst_h_rdc0 can read tst_netcdf4_c0.nc. 2022-11-21T22:42:54.9610229Z 2022-11-21T22:42:54.9610601Z *** Checking HDF5 file c0.nc. 2022-11-21T22:42:54.9611028Z *** Checking HDF5 objcts...ok. 2022-11-21T22:42:54.9611448Z *** Tests successful! 2022-11-21T22:42:54.9611900Z *** Running tst_create_files.c to create test files. 2022-11-21T22:42:54.9612197Z 2022-11-21T22:42:54.9612567Z *** Creating test files for ncdump. 2022-11-21T22:42:54.9614238Z *** creating nested group file tst_solar_1.nc...Sorry! Unexpected result, $SRC_DIR/ncdump/tst_create_files.c, line: 71 2022-11-21T22:42:54.9719951Z 58: $SRC_DIR/build-shared/ncdump/ncdump: tst_solar_1.nc: NetCDF: Unknown file format 2022-11-21T22:42:54.9748238Z 50/229 Test #58: ncdump_tst_nccopy4 ....................***Failed 0.42 sec |
Full log https://dev.azure.com/conda-forge/84710dde-1620-425b-80d0-4cf5baca359d/_apis/build/builds/616373/logs/64 is available if needed; here I snipped out unrelated tests.
Again, the fact that the tests failing are running concurrently is suspicious, so poke around and find that both |ncdump/tst_netcdf4.sh| and |ncdump/tst_nccopy4.sh| reference |tst_solar_1.nc| (and cdl) as well as |tst_solar_2.nc| (and cdl).
I also see mention of |tst_solar| across the codebase:
|ncdump/tst_create_files.c 16:#define FILE_NAME_1 "tst_solar_1.nc" 17:#define FILE_NAME_2 "tst_solar_2.nc" 18:#define FILE_NAME_CMP "tst_solar_cmp.nc" ncdump/tst_nccopy4.sh 23: tst_group_data tst_nans tst_opaque_data tst_solar_1 tst_solar_2 24: tst_solar_cmp tst_special_atts' ncdump/tst_ncgen_shared.sh 90:ref_tst_solar_1 \ 91:ref_tst_solar_2 \ 140:BIGTESTS4="ref_tst_solar_1" ncdump/tst_netcdf4.sh 47:${NCDUMP} tst_solar_1.nc | sed 's/e+0/e+/g' > tst_solar_1.cdl ; ERR 48:diff -b tst_solar_1.cdl $srcdir/ref_tst_solar_1.cdl ; ERR 49:${NCDUMP} tst_solar_2.nc | sed 's/e+0/e+/g' > tst_solar_2.cdl ; ERR 50:diff -b tst_solar_2.cdl $srcdir/ref_tst_solar_2.cdl ; ERR ncgen/ctests.sh 84:ref_tst_solar_1 \ 85:ref_tst_solar_2 \ 104:BIGTESTS4="ref_tst_solar_1" ncgen/ftests.sh 85:ref_tst_solar_1 \ 86:ref_tst_solar_2 \ 105:BIGTESTS4="ref_tst_solar_1" ncgen/jtests.sh 84:ref_tst_solar_1 \ 85:ref_tst_solar_2 \ 108:BIGTESTS4="ref_tst_solar_1" nczarr_test/run_ncgen4.sh |
Is there a better way to approach this testing than playing "whack a mole" by manually renaming files? This is proving pretty brittle.
— Reply to this email directly, view it on GitHub https://github.com/Unidata/netcdf-c/issues/2562, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG47W5VV5RD4E5KHXO7NT3WJP6BFANCNFSM6AAAAAASHE2OM4. You are receiving this because you are subscribed to this thread.Message ID: @.***>
If you used a random temp directory for the intermediate test output that would let the scripts avoid stomping on each other.
Adding another case here, there's also a conflict between those two same test scripts and tst_special_atts:
2022-11-21T23:34:24.6009450Z 58: Test command: /usr/bin/bash "-c" "export srcdir=$SRC_DIR/ncdump;export TOPSRCDIR=$SRC_DIR;$SRC_DIR/build-shared/ncdump/tst_nccopy4.sh tst_nccopy4.sh "
2022-11-21T23:34:24.6010479Z 58: Working Directory: $SRC_DIR/build-shared/ncdump
2022-11-21T23:34:24.6011055Z 58: Test timeout computed to be: 1500
2022-11-21T23:34:24.6260448Z 58:
2022-11-21T23:34:24.6263645Z 58: *** Testing groups.
2022-11-21T23:34:24.6269002Z 58: *** creating nested group file tst_group_data.nc...ok.
2022-11-21T23:34:24.6278758Z 58: *** Tests successful!
2022-11-21T23:34:24.6380762Z 58:
2022-11-21T23:34:24.6385867Z 58: *** Testing enums.
2022-11-21T23:34:24.6391021Z 58: *** creating enum test file tst_enum_data.nc...ok.
2022-11-21T23:34:24.6396102Z 58: *** Tests successful!
2022-11-21T23:34:24.6500524Z 58:
2022-11-21T23:34:24.6505633Z 58: *** Testing compound types.
2022-11-21T23:34:24.6510648Z 58: *** creating compound test file tst_comp.nc...ok.
2022-11-21T23:34:24.6516878Z 58: *** Tests successful!
2022-11-21T23:34:24.6619978Z 58:
2022-11-21T23:34:24.6625170Z 58: *** Testing compound types some more.
2022-11-21T23:34:24.6630250Z 58: *** creating another compound test file tst_comp2.nc...ok.
2022-11-21T23:34:24.6635354Z 58: *** Tests successful!
2022-11-21T23:34:24.6641234Z 58:
2022-11-21T23:34:24.6646837Z 58: *** Testing netCDF-4 features of nccopy on ncdump/*.nc files
2022-11-21T23:34:24.6652088Z 58: *** Test nccopy tst_comp.nc copy_of_tst_comp.nc ...
2022-11-21T23:34:24.6992973Z 58: *** compare with copy_of_tst_comp.cdl
2022-11-21T23:34:24.7015943Z 58: *** Test nccopy tst_comp2.nc copy_of_tst_comp2.nc ...
2022-11-21T23:34:24.7371729Z 58: *** compare with copy_of_tst_comp2.cdl
2022-11-21T23:34:24.7397609Z 58: *** Test nccopy tst_enum_data.nc copy_of_tst_enum_data.nc ...
2022-11-21T23:34:24.7738782Z 58: *** compare with copy_of_tst_enum_data.cdl
2022-11-21T23:34:24.7764467Z 58: *** Test nccopy tst_fillbug.nc copy_of_tst_fillbug.nc ...
2022-11-21T23:34:24.8158988Z 58: *** compare with copy_of_tst_fillbug.cdl
2022-11-21T23:34:24.8187225Z 58: *** Test nccopy tst_group_data.nc copy_of_tst_group_data.nc ...
2022-11-21T23:34:24.8659300Z 58: *** compare with copy_of_tst_group_data.cdl
2022-11-21T23:34:24.8688832Z 58: *** Test nccopy tst_nans.nc copy_of_tst_nans.nc ...
2022-11-21T23:34:24.9063995Z 58: *** compare with copy_of_tst_nans.cdl
2022-11-21T23:34:24.9190838Z 58: *** Test nccopy tst_opaque_data.nc copy_of_tst_opaque_data.nc ...
2022-11-21T23:34:24.9569293Z 58: *** compare with copy_of_tst_opaque_data.cdl
2022-11-21T23:34:24.9600107Z 58: *** Test nccopy tst_solar_1.nc copy_of_tst_solar_1.nc ...
2022-11-21T23:34:25.0017715Z 58: *** compare with copy_of_tst_solar_1.cdl
2022-11-21T23:34:25.0046940Z 58: *** Test nccopy tst_solar_2.nc copy_of_tst_solar_2.nc ...
2022-11-21T23:34:25.0522267Z 58: *** compare with copy_of_tst_solar_2.cdl
2022-11-21T23:34:25.0555174Z 58: *** Test nccopy tst_solar_cmp.nc copy_of_tst_solar_cmp.nc ...
2022-11-21T23:34:25.0917444Z 58: *** compare with copy_of_tst_solar_cmp.cdl
2022-11-21T23:34:25.0949565Z 58: *** Test nccopy tst_special_atts.nc copy_of_tst_special_atts.nc ...
2022-11-21T23:34:25.1774438Z 58: *** compare with copy_of_tst_special_atts.cdl
2022-11-21T23:34:25.1806127Z 58: *** Test nccopy tst_string_data.nc copy_of_tst_string_data.nc ...
2022-11-21T23:34:25.2180606Z 58: *** compare with copy_of_tst_string_data.cdl
2022-11-21T23:34:25.2322256Z 58: *** Creating compressible test files tst_inflated.nc, tst_inflated4.nc...ok.
2022-11-21T23:34:25.2328177Z 58: *** Tests successful!
2022-11-21T23:34:25.2341515Z 58: *** Test nccopy -d1 can compress a classic format file ...
2022-11-21T23:34:25.2537353Z 58: *** Test nccopy -d1 can compress a netCDF-4 format file ...
2022-11-21T23:34:25.2742467Z 58: *** Test nccopy -d1 -s can compress a classic model netCDF-4 file even more ...
2022-11-21T23:34:25.2934604Z 58: *** Test nccopy -d1 -s can compress a netCDF-4 file even more ...
2022-11-21T23:34:25.3133357Z 58: *** Test nccopy -d0 turns off compression, shuffling of compressed, shuffled file ...
2022-11-21T23:34:25.3514806Z 58: *** Testing nccopy -d1 -s on ncdump/*.nc files
2022-11-21T23:34:25.3523050Z 58: *** Test nccopy -d1 -s tst_comp.nc copy_of_tst_comp.nc ...
2022-11-21T23:34:25.3933518Z 58: *** Test nccopy -d1 -s tst_comp2.nc copy_of_tst_comp2.nc ...
2022-11-21T23:34:25.4348148Z 58: *** Test nccopy -d1 -s tst_enum_data.nc copy_of_tst_enum_data.nc ...
2022-11-21T23:34:25.4383908Z test 42
2022-11-21T23:34:25.4389668Z Start 42: ncdump_tst_netcdf4
2022-11-21T23:34:25.4395601Z
2022-11-21T23:34:25.4401536Z 42: Test command: /usr/bin/bash "-c" "export srcdir=$SRC_DIR/ncdump;export TOPSRCDIR=$SRC_DIR;$SRC_DIR/build-shared/ncdump/tst_netcdf4.sh tst_netcdf4.sh "
2022-11-21T23:34:25.4407426Z 42: Working Directory: $SRC_DIR/build-shared/ncdump
2022-11-21T23:34:25.4412454Z 42: Test timeout computed to be: 1500
2022-11-21T23:34:25.4516725Z 42:
2022-11-21T23:34:25.4522821Z 42: *** Testing ncgen and ncdump for netCDF-4 format.
2022-11-21T23:34:25.4832883Z 58: *** Test nccopy -d1 -s tst_fillbug.nc copy_of_tst_fillbug.nc ...
2022-11-21T23:34:25.5268063Z 58: *** Test nccopy -d1 -s tst_group_data.nc copy_of_tst_group_data.nc ...
2022-11-21T23:34:25.5276009Z 42: *** Creating test output tst_netcdf4_c0.nc.
2022-11-21T23:34:25.5615932Z 42: *** Testing that program tst_h_rdc0 can read tst_netcdf4_c0.nc.
2022-11-21T23:34:25.5739026Z 42:
2022-11-21T23:34:25.5744828Z 42: *** Checking HDF5 file c0.nc.
2022-11-21T23:34:25.5750305Z 42: *** Checking HDF5 objcts...ok.
2022-11-21T23:34:25.5755516Z 42: *** Tests successful!
2022-11-21T23:34:25.5761185Z 42: *** Running tst_create_files.c to create test files.
2022-11-21T23:34:25.5794460Z 58: *** Test nccopy -d1 -s tst_nans.nc copy_of_tst_nans.nc ...
2022-11-21T23:34:25.5875292Z 42:
2022-11-21T23:34:25.5880682Z 42: *** Creating test files for ncdump.
2022-11-21T23:34:25.5886249Z 42: *** creating nested group file tst_solar_1.nc...ok.
2022-11-21T23:34:25.5891379Z 42: *** checking nested group file tst_solar_1.nc...ok.
2022-11-21T23:34:25.5896494Z 42: *** creating file with VLEN tst_solar_2.nc...ok.
2022-11-21T23:34:25.5901544Z 42: *** creating file with compound type tst_solar_cmp.nc...ok.
2022-11-21T23:34:25.5906589Z 42: *** Tests successful!
2022-11-21T23:34:25.5912247Z 42: *** Testing tst_create_files output for netCDF-4 features.
2022-11-21T23:34:25.6180785Z 42: *** Running tst_group_data.c to create test files.
2022-11-21T23:34:25.6271941Z 58: *** Test nccopy -d1 -s tst_opaque_data.nc copy_of_tst_opaque_data.nc ...
2022-11-21T23:34:25.6295128Z 42:
2022-11-21T23:34:25.6300244Z 42: *** Testing groups.
2022-11-21T23:34:25.6305383Z 42: *** creating nested group file tst_group_data.nc...ok.
2022-11-21T23:34:25.6310555Z 42: *** Tests successful!
2022-11-21T23:34:25.6454710Z 42: *** Testing -v option with absolute name and groups...
2022-11-21T23:34:25.6601959Z 42: *** Testing -v option with relative name and groups...
2022-11-21T23:34:25.6689383Z 58: *** Test nccopy -d1 -s tst_solar_1.nc copy_of_tst_solar_1.nc ...
2022-11-21T23:34:25.6747924Z 42: *** Running tst_enum_data.c to create test files.
2022-11-21T23:34:25.6856774Z 42:
2022-11-21T23:34:25.6864111Z 42: *** Testing enums.
2022-11-21T23:34:25.6869725Z 42: *** creating enum test file tst_enum_data.nc...ok.
2022-11-21T23:34:25.6874976Z 42: *** Tests successful!
2022-11-21T23:34:25.7009676Z 42: *** Running tst_enum_undef.c to create test files.
2022-11-21T23:34:25.7119897Z 42:
2022-11-21T23:34:25.7125842Z 42: *** Testing enum undefined identifier.
2022-11-21T23:34:25.7132942Z 42: *** creating enum test file tst_enum_undef.nc...ok.
2022-11-21T23:34:25.7138092Z 42: *** Tests successful!
2022-11-21T23:34:25.7165520Z 58: *** Test nccopy -d1 -s tst_solar_2.nc copy_of_tst_solar_2.nc ...
2022-11-21T23:34:25.7270360Z 42: *** Running tst_opaque_data.c to create test files.
2022-11-21T23:34:25.7380853Z 42:
2022-11-21T23:34:25.7386317Z 42: *** Testing opaque types.
2022-11-21T23:34:25.7391661Z 42: *** creating opaque test file tst_opaque_data.nc...ok.
2022-11-21T23:34:25.7397122Z 42: *** Tests successful!
2022-11-21T23:34:25.7524427Z 42: *** Running tst_comp.c to create test files.
2022-11-21T23:34:25.7552487Z 58: *** Test nccopy -d1 -s tst_solar_cmp.nc copy_of_tst_solar_cmp.nc ...
2022-11-21T23:34:25.7637060Z 42:
2022-11-21T23:34:25.7642209Z 42: *** Testing compound types.
2022-11-21T23:34:25.7647397Z 42: *** creating compound test file tst_comp.nc...ok.
2022-11-21T23:34:25.7653853Z 42: *** Tests successful!
2022-11-21T23:34:25.7781734Z 42: *** Running tst_nans.c to create test files.
2022-11-21T23:34:25.7869488Z 42:
2022-11-21T23:34:25.7874747Z 42: *** Testing NaN
2022-11-21T23:34:25.7879949Z 42: *** creating NaN test file tst_nans.nc...ok.
2022-11-21T23:34:25.7885101Z 42: *** Tests successful!
2022-11-21T23:34:25.7958740Z 58: *** Test nccopy -d1 -s tst_special_atts.nc copy_of_tst_special_atts.nc ...
2022-11-21T23:34:25.8002764Z 42: *** Running tst_special_atts.c to create test files.
2022-11-21T23:34:25.8083812Z 42:
2022-11-21T23:34:25.8089963Z 42: *** Testing '-s' option for special attributes.
2022-11-21T23:34:25.8095703Z 42: *** creating special attributes test file tst_special_atts.nc...Sorry! Unexpected result, $SRC_DIR/ncdump/tst_special_atts.c, line: 61
2022-11-21T23:34:25.8102488Z 49/232 Test #42: ncdump_tst_netcdf4 ....................***Failed 0.37 sec
2022-11-21T23:34:25.8107517Z
And then another with tst_group_data in ncdump:
2022-11-21T23:34:22.6601112Z 58: Test command: /usr/bin/bash "-c" "export srcdir=$SRC_DIR/ncdump;export TOPSRCDIR=$SRC_DIR;$SRC_DIR/build-shared/ncdump/tst_nccopy4.sh tst_nccopy4.sh "
2022-11-21T23:34:22.6602041Z 58: Working Directory: $SRC_DIR/build-shared/ncdump
2022-11-21T23:34:22.6603354Z 58: Test timeout computed to be: 1500
<SNIP>
2022-11-21T23:34:22.9261052Z
2022-11-21T23:34:22.9350171Z 16: Test command: $SRC_DIR/build-shared/ncdump/tst_group_data
2022-11-21T23:34:22.9351154Z 16: Working Directory: $SRC_DIR/build-shared/ncdump
2022-11-21T23:34:22.9351627Z 16: Test timeout computed to be: 1500
2022-11-21T23:34:22.9362224Z 58: $SRC_DIR/build-shared/ncdump/ncdump: tst_group_data.nc: NetCDF: HDF error
2022-11-21T23:34:22.9380723Z 32/234 Test #58: ncdump_tst_nccopy4 ....................***Failed 0.28 sec
Ah, I see the reason we don't see such issues here isn't that we're not relying on it, it's that they're run with --rerun-failed (at least that's why I assume we don't see them here).
I think we can kill the interference cases. The real problem is that some tests depend on the output of other tests as their input.
As long as we know which order the tests need to be run in, the mechanisms to ensure "test A" finishes before "test B" is run exist in both autotools and cmake. We already make copious use of them. So adding more shouldn't be a problem as we establish what depends on what.
I'd also argue that relying on side effects of tests instead of having that as an explicit separate dependency (e.g. a fixture/setup) is a problem with the tests that should at some point be addressed rather than continually worked around.
Just because they're tests doesn't mean they're any less deserving of programmers' love. 😆