"nc_test/test_byterange.sh" appears to be broken under HDF5 1.12.0

[WardF]

The test nc_test/test_byterange.sh is failing on my system. It appears that this feature is not being tested by our github actions. I will work to establish the fail states for this test (is it universally broken, or is it related to the environment? Thus far it has only been tested on OSX M1 hardware w/ autotools). I will also wire this in to our Github Actions.

Also tagging in @DennisHeimbigner.

Update:

Forgot to add a pertinent detail; the issue first arose using HDF5 1.12.0. Testing with HDF5 1.10.7 does not demonstrate the error.

The error observed is as follows:

***Test remote netdf-4 file: s3
Assertion failed: (0 == memcmp(block->b + 8 + block->u.info.size,H5MM_block_tail_guard_s,8)), function H5MM__sanity_check_block, file H5MM.c, line 165.
Assertion failed: (0 == memcmp(block->b + 8 + block->u.info.size,H5MM_block_tail_guard_s,8)), function H5MM__sanity_check_block, file H5MM.c, line 165.
/Users/wfisher/Desktop/gitprojects/netcdf-c/build-12/nc_test/test_byterange.sh: line 25: 64256 Abort trap: 6           ${NCDUMP} -h "$U" > tst_http_$TAG.cdl

[DennisHeimbigner]

I will check also. My speculation is that one of the URLs it is using is broken.

[WardF]

@DennisHeimbigner I updated the details above; it appears related somehow perhaps to HDF5 1.12.0 vs. 1.10.7.

[DennisHeimbigner]

Ok, then the virtual driver we are using has probably changed on 1.12. I will investigate.

[DennisHeimbigner]

Looking at the cmake actions failure, I see this:

/home/runner/work/netcdf-c/netcdf-c/libhdf5/H5FDhttp.c:462:19: error: ‘H5FD_FEAT_DEFAULT_VFD_COMPATIBLE’ undeclared (first use in this function)

Looks like something changed. I will see if I can run it down.

[DennisHeimbigner]

Weird. That macro is defined both in 1.10.7 and 1.12.0 in exactly the same .h file. Why is it not seeing it?

[WardF]

I'm observing the following when running with 1.12.0:

 73/211 Test  #73: nc_test_test_byterange ................***Failed   21.02 sec

*** Testing reading NetCDF-3 file with http
***Test remote classic file
***Test remote netdf-4 file: s3
Assertion failed: (0 == memcmp(block->b + 8 + block->u.info.size,H5MM_block_tail_guard_s,8)), function H5MM__sanity_check_block, file H5MM.c, line 165.
Assertion failed: (0 == memcmp(block->b + 8 + block->u.info.size,H5MM_block_tail_guard_s,8)), function H5MM__sanity_check_block, file H5MM.c, line 165.
/Users/wfisher/Desktop/gitprojects/netcdf-c/build-12/nc_test/test_byterange.sh: line 25: 64256 Abort trap: 6           ${NCDUMP} -h "$U" > tst_http_$TAG.cdl

[WardF]

Don't trust the Github Actions yet, we aren't testing against 1.12 there (yet), so I would expect them to pass. Can you recreate the issue locally?

[WardF]

Refresh my memory if you would, @DennisHeimbigner, is there a minimum required hdf5 version for byterange support?

[DennisHeimbigner]

Don't remember about min version. I will test under linux on ubuntu 16.

[DennisHeimbigner]

So, I ran the test under unbuntu 16 with 1.12.0 with the 18 compatibility flag. It did not fail. Could this be a return of the bash vs sh issue?

[WardF]

Possibly. The failure in github actions is specific to HDF5 1.10.1, so we can ignore that for the moment. I'll try to figure out what's going on, it might also be architecture-dependent.

[WardF]

Rebuilding HDF5 1.12.0 with debugging and sanitization enabled to try to make sense of this; also need to test on x64 architecture, the failures thus far have been on ARM.