Unidata
Fails to build with HDF5 2.0.0
I tried to build netcdf 4.9.3 with the latest HDF5 2.0.0 but it fails:
[102/243] Building C object libhdf5/CMakeFiles/netcdfhdf5.dir/H5FDhttp.c.obj
FAILED: [code=1] libhdf5/CMakeFiles/netcdfhdf5.dir/H5FDhttp.c.obj
C:\msys64\clang64\bin\clang.exe -DH5_BUILT_AS_DYNAMIC_LIB -DHAVE_CONFIG_H -ID:/_/B/src/build-static-CLANG64/include -ID:/_/B/src/netcdf-c-4.9.3/include -ID:/_/B/src/netcdf-c-4.9.3/oc2 -ID:/_/B/src/netcdf-c-4.9.3/libsrc -ID:/_/B/src/build-static-CLANG64 -march=nocona -msahf -mtune=generic -O2 -pipe -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong -Wp,-D__USE_MINGW_ANSI_STDIO=1 -Wno-sign-conversion -Wno-float-conversion -Wno-incompatible-pointer-types -DLIBXML_STATIC -fmacro-prefix-map='D:/_/B/src/netcdf-c-4.9.3'=. -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG -MD -MT libhdf5/CMakeFiles/netcdfhdf5.dir/H5FDhttp.c.obj -MF libhdf5\CMakeFiles\netcdfhdf5.dir\H5FDhttp.c.obj.d -o libhdf5/CMakeFiles/netcdfhdf5.dir/H5FDhttp.c.obj -c D:/_/B/src/netcdf-c-4.9.3/libhdf5/H5FDhttp.c
D:/_/B/src/netcdf-c-4.9.3/libhdf5/H5FDhttp.c:57:2: error: "Cannot determine version of H5FD_class_t"
57 | #error "Cannot determine version of H5FD_class_t"
| ^
D:/_/B/src/netcdf-c-4.9.3/libhdf5/H5FDhttp.c:316:35: error: call to undeclared function 'H5FDperform_init'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
316 | return H5Pset_driver(fapl_id, H5FD_HTTP, NULL);
| ^
D:/_/B/src/netcdf-c-4.9.3/libhdf5/H5FDhttp.h:34:20: note: expanded from macro 'H5FD_HTTP'
34 | #define H5FD_HTTP (H5FDperform_init(H5FD_http_init))
| ^
2 errors generated.
Looking at the code It looks like It accepts only HDF5 1.x version https://github.com/Unidata/netcdf-c/blob/v4.9.3/libhdf5/H5FDhttp.c#L52 but It's 2 in HDF5 https://github.com/HDFGroup/hdf5/blob/hdf5_2.0.0/src/H5public.h#L64
Would be nice if NetCDF's CI had a job to build against HDF5 from git master, say every night. Then issues can be found before HDF releases.
I see there is a public H5FD_CLASS_VERSION macro since HDF5 1.14.1. Would this do?
diff --git a/libhdf5/H5FDhttp.c b/libhdf5/H5FDhttp.c
index 917c73a..ed17fd3 100644
--- a/libhdf5/H5FDhttp.c
+++ b/libhdf5/H5FDhttp.c
@@ -53,6 +53,8 @@ Define a simple #ifdef test for the version of H5FD_class_t we are using
#if H5_VERS_MINOR < 10
#define H5FDCLASS1 1
#endif
+#elif defined H5FD_CLASS_VERSION && H5FD_CLASS_VERSION == 1
+/* since HDF5 1.14.1, including at least 2.0.0 : same as >= 1.10 */
#else
#error "Cannot determine version of H5FD_class_t"
#endif
Edit: Yes, netcdf-c and hdf5 do not agree on what is "version 1".
I see there is a public
H5FD_CLASS_VERSIONmacro since HDF5 1.14.1. Would this do?diff --git a/libhdf5/H5FDhttp.c b/libhdf5/H5FDhttp.c index 917c73a..ed17fd3 100644 --- a/libhdf5/H5FDhttp.c +++ b/libhdf5/H5FDhttp.c @@ -53,6 +53,8 @@ Define a simple #ifdef test for the version of H5FD_class_t we are using #if H5_VERS_MINOR < 10 #define H5FDCLASS1 1 #endif +#elif defined H5FD_CLASS_VERSION && H5FD_CLASS_VERSION == 1 +/* since HDF5 1.14.1, including at least 2.0.0 : same as >= 1.10 */ #else #error "Cannot determine version of H5FD_class_t" #endif Edit: Yes, netcdf-c and hdf5 do not agree on what is "version 1".
This is not enough to build netcdf-c against HDF5 2.0.0
D:/_/B/src/netcdf-c-4.9.3/libhdf5/H5FDhttp.h:34:26: error: implicit declaration of function 'H5FDperform_init' [-Wimplicit-function-declaration]
34 | #define H5FD_HTTP (H5FDperform_init(H5FD_http_init))
| ^~~~~~~~~~~~~~~~
H5FDperform_init was removed in HDF5 2.0.0
Indeed.
Removed
H5FDperform_initAPI routine.Virtual File Driver (VFD) developers who wish to provide an ID for their driver should create a routine specific to their individual implementation.
https://github.com/HDFGroup/hdf5/commit/2c58357b8f8d9f4dffad5859816c9ed8c0a8216b
I reported the missing H5FDperform_init upstream. Cases like this in the netCDF library should be handled by HDF5 compatibility macros to the greatest extent possible, so as to minimize the need for version patching.
https://forum.hdfgroup.org/t/2-0-missing-api-compatibility-macros/13599
I just ran a quick survey in the netCDF-C 4.9.3 source code. It looks like H5FDperform_init is the only missing compatibility macro that is needed for 4.9.3. The other four are not needed here, to fix build for HDF5 2.0.0.
I confirm that that only netcdf-versions 4.9.2 and 4.9.3 are affected by the now-missing function H5FDperform_init. This function call was added shortly before 4.9.2 release in commit https://github.com/Unidata/netcdf-c/commit/727fa749005fa7868735fff82474b4476c6f5c6f.
Also, for what it's worth, HDF5 upstream has confirmed that the now-missing function was officially in the public API before HDF5 2.0.0.
Looking at this, they seem to have made this a lot more complex See section entitled Creating and Opening Files
I'm working on this now, as I've gotten 2.0.0 wired into my provisioning and test infrastructure. The removal of autotools highlighted a number of changes I needed to make in my scripts.
I've dropped a note in to the HDF Forums asking for some clarification on their plans.
You can try changing that line in H5FDhttp.h to #define H5FD_HTTP (H5OPEN H5FD_http_init())
This should work with any version of HDF5 and you may be able to eliminate the code that does something else for versions < 1.13.2 and just always use the above.
If for some reason this doesn't work with 1.14, you should be able to extend the preprocessor code in that function that uses H5FDperform_init if the version is greater than or equal to 1.13.2 to also avoid using it if the version is equal to or greater than 2.0.0. The commit in question essentially reverted the file driver initialization method to that used before 1.14.
Thanks @fortnern, this looks promising, I'm working on testing this fix right now. Thanks!
Thanks @fortnern this fix worked, but now I'm seeing some issues with the szip tests in netcdf-fortran, so I'll need to sort those out as well :). But thanks, we're getting closer!