Review HDF5 iosp for superblock version 3 features, gather example files.

[JohnLCaron]

HDF5 has a "version 3" spec. Our code should be reviewed for conformance.

We especially need to gather example files that use version 3 features for testing.

https://support.hdfgroup.org/HDF5/doc/H5.format.html

[lesserwhirls]

I have a netCDF-4 file that was written with superblock 3, and its equivalent using super block 0 (a little background on how these came about can be found here). To me this points to a serious flaw in that the netCDF-4 persistence layer is ill defined (outside of "see HDF5 for details"). This is where @oxelson and I were going to start (both attached here), so that at the very least we could read netCDF-4 files using HDF Superblock 3. Thankfully, there should be very few netCDF-4 files like this in the wild, however.

In terms of files to test out support for the new features, I'm thinking the best way would be for me to reach out to the HDF5 group to see if they have some samples using those features. They seem open to helping out (see https://github.com/Unidata/netcdf-c/issues/1545). It's possible their test suite produces some files that could be of use, so that would be another place to look. I'll be out of town until next week, but can take a look then.

nc4-sb0-3.zip

[epourmal]

Yes, we can provide you with such files. Please shoot email to [email protected] with this request.

[Yaqiang]

Any progress on this issue?

[rschmunk]

Checking on this again a year later. I just heard from someone at JPL about a dataset created using "HDF5 version 1.14.3" that could not be read, and the problem is that

Caused by: java.io.IOException: Unknown superblock version= 3 at ucar.nc2.internal.iosp.hdf5.H5headerNew.read(H5headerNew.java:257) at ucar.nc2.internal.iosp.hdf5.H5iospNew.build(H5iospNew.java:126) ...