Unidata
Unresolved external using static build on Windows
(EDIT: Sorry, pressed a keyboard shortcut to post, before finishing)
I'm trying to build NetCDF and its dependencies on Windows as static libraries, for use with Exodus. When I link my application (manually listing the static libraries in many different orders) I'm getting the error messages shown below about unresolved external symbols __imp_H5DS, etc.
I noticed that these symbols are present in netcdf.lib
(base) PS > dumpbin /symbols netcdf.lib | Select-String -Pattern H5DS
05E 00000000 UNDEF notype External | __imp_H5Dset_extent
07A 00000000 UNDEF notype External | __imp_H5DSget_num_scales
07B 00000000 UNDEF notype External | __imp_H5DSget_scale_name
07C 00000000 UNDEF notype External | __imp_H5DSis_scale
07D 00000000 UNDEF notype External | __imp_H5DSiterate_scales
038 00000000 UNDEF notype External | __imp_H5DSdetach_scale
072 00000000 UNDEF notype External | __imp_H5Dset_extent
097 00000000 UNDEF notype External | __imp_H5DSattach_scale
098 00000000 UNDEF notype External | __imp_H5DSdetach_scale
099 00000000 UNDEF notype External | __imp_H5DSset_scale
but lack the __imp_ prefix in libhdf5_hl.lib.
(base) PS > dumpbin /symbols libhdf5_hl.lib | Select-String -Pattern H5DS
056 00000000 UNDEF notype () External | H5Dset_extent
04B 00000000 SECTF notype () External | H5DSwith_new_ref
04C 00000000 SECT5 notype () External | H5DSattach_scale
04D 00000000 SECT6 notype () External | H5DSdetach_scale
04E 00000000 SECTE notype () External | H5DSset_scale
04F 00000000 SECT8 notype () External | H5DSget_num_scales
050 00000000 SECTD notype () External | H5DSset_label
051 00000000 SECT7 notype () External | H5DSget_label
052 00000000 SECT9 notype () External | H5DSget_scale_name
053 00000000 SECTB notype () External | H5DSis_scale
054 00000000 SECTC notype () External | H5DSiterate_scales
055 00000000 SECTA notype () External | H5DSis_attached
05B 00000000 SECT4 notype () Static | H5DS_is_reserved
Is there something I'm doing obviously wrong? Does anyone have a working example of building + using NetCDF on Windows using Powershell + CMake + MSVC only (trying to avoid MINGW, etc.)
Linker errors
netcdf.lib(hdf5open.obj) : error LNK2019: unresolved external symbol __imp_H5DSget_num_scales referenced in function get_attached_info
netcdf.lib(hdf5open.obj) : error LNK2019: unresolved external symbol __imp_H5DSget_scale_name referenced in function read_scale
netcdf.lib(hdf5open.obj) : error LNK2019: unresolved external symbol __imp_H5DSis_scale referenced in function read_hdf5_obj
netcdf.lib(hdf5open.obj) : error LNK2019: unresolved external symbol __imp_H5DSiterate_scales referenced in function get_attached_info
netcdf.lib(nc4info.obj) : error LNK2019: unresolved external symbol __imp_H5Tset_cset referenced in function NC4_write_ncproperties
netcdf.lib(nc4hdf.obj) : error LNK2001: unresolved external symbol __imp_H5Tset_cset
netcdf.lib(nc4info.obj) : error LNK2019: unresolved external symbol __imp_H5Tset_strpad referenced in function NC4_write_ncproperties
netcdf.lib(nc4hdf.obj) : error LNK2001: unresolved external symbol __imp_H5Tset_strpad
netcdf.lib(nc4info.obj) : error LNK2019: unresolved external symbol __imp_H5Acreate1 referenced in function NC4_write_ncproperties
netcdf.lib(nc4hdf.obj) : error LNK2001: unresolved external symbol __imp_H5Acreate1
netcdf.lib(hdf5internal.obj) : error LNK2019: unresolved external symbol __imp_H5DSdetach_scale referenced in function nc4_reform_coord_var
netcdf.lib(nc4hdf.obj) : error LNK2001: unresolved external symbol __imp_H5DSdetach_scale
netcdf.lib(nc4hdf.obj) : error LNK2019: unresolved external symbol __imp_H5Tcommit1 referenced in function commit_type
netcdf.lib(nc4hdf.obj) : error LNK2019: unresolved external symbol __imp_H5Aget_num_attrs referenced in function NC4_walk
netcdf.lib(nc4hdf.obj) : error LNK2019: unresolved external symbol __imp_H5Aopen_idx referenced in function NC4_walk
netcdf.lib(nc4hdf.obj) : error LNK2019: unresolved external symbol __imp_H5Dopen1 referenced in function NC4_walk
netcdf.lib(nc4hdf.obj) : error LNK2019: unresolved external symbol __imp_H5Pget_version referenced in function NC4_hdf5get_superblock
netcdf.lib(nc4hdf.obj) : error LNK2019: unresolved external symbol __imp_H5DSattach_scale referenced in function nc4_rec_write_metadata
netcdf.lib(nc4hdf.obj) : error LNK2019: unresolved external symbol __imp_H5DSset_scale referenced in function nc4_create_dim_wo_var
netcdf.lib(hdf5plugins.obj) : error LNK2019: unresolved external symbol __imp_H5PLappend referenced in function NC4_hdf5_plugin_path_set
netcdf.lib(hdf5plugins.obj) : error LNK2019: unresolved external symbol __imp_H5PLremove referenced in function NC4_hdf5_plugin_path_set
netcdf.lib(hdf5plugins.obj) : error LNK2019: unresolved external symbol __imp_H5PLget referenced in function NC4_hdf5_plugin_path_get
netcdf.lib(hdf5plugins.obj) : error LNK2019: unresolved external symbol __imp_H5PLsize referenced in function NC4_hdf5_plugin_path_get
Script I used to build NetCDF + dependencies as static libraries
# Specify installation path
$Env:INSTALL_PATH="C:/Users/username/Documents/install"
#
#
# libaec
#
curl https://gitlab.dkrz.de/k202009/libaec/-/archive/v1.1.3/libaec-v1.1.3.zip -o libaec.zip
Expand-Archive .\libaec.zip
cd libaec
cd libaec-v1.1.3
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX="$Env:INSTALL_PATH" -DBUILD_TESTING=OFF ..
cmake --build . --target install --config Release
cd ../../..
#
# zlib
#
curl https://github.com/madler/zlib/archive/refs/tags/v1.3.1.zip -o zlib.zip
Expand-Archive zlib.zip
cd zlib
cd zlib-1.3.1
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX="$Env:INSTALL_PATH" -DZLIB_BUILD_EXAMPLES=OFF ..
cmake --build . --target install --config Release
cd ../../..
#
# HDF5
#
curl https://github.com/HDFGroup/hdf5/archive/refs/tags/hdf5_1.14.6.zip -o hdf5.zip
Expand-Archive .\hdf5.zip
cd hdf5
cd hdf5-hdf5_1.14.6
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX="$Env:INSTALL_PATH" -DBUILD_TESTING=OFF -DHDF5_BUILD_EXAMPLES=OFF -DHDF5_BUILD_UTILS=OFF -DHDF5_BUILD_TOOLS=OFF -DBUILD_SHARED_LIBS=OFF -DBUILD_STATIC_LIBS=ON -DSZIP_LIBRARY="$Env:INSTALL_PATH"/lib/szip-static.lib -DSZIP_INCLUDE_DIR="$Env:INSTALL_PATH"/include -DZLIB_LIBRARY="$Env:INSTALL_PATH"/lib/zlibstatic.lib -DZLIB_INCLUDE_DIR="$Env:INSTALL_PATH"/include -DHDF5_USE_LIBAEC_STATIC=ON -Dlibaec_DIR="$Env:INSTALL_PATH"/cmake -Dlibaec_INCLUDE_DIR="$Env:INSTALL_PATH"/include -Dlibaec_LIBRARY="$Env:INSTALL_PATH"/lib/aec-static.lib ..
cmake --build . --target install --config Release
cd ../../..
#
# Netcdf
#
curl https://github.com/Unidata/netcdf-c/archive/refs/tags/v4.9.3.zip -o netcdf.zip
Expand-Archive .\netcdf.zip
cd netcdf
cd netcdf-c-4.9.3
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX="$Env:INSTALL_PATH" -DBUILD_SHARED_LIBS=OFF -DBUILD_TESTING=OFF -DBUILD_TESTSETS=OFF -DNETCDF_ENABLE_TESTS=OFF -DNETCDF_ENABLE_UNIT_TESTS=OFF -DNETCDF_ENABLE_FILTER_TESTING=OFF -DNETCDF_ENABLE_FILTER_ZSTD=OFF -DNETCDF_ENABLE_FILTER_BZ2=OFF -DNETCDF_ENABLE_DAP=OFF -DNETCDF_ENABLE_BYTERANGE=OFF -DNETCDF_ENABLE_NCZARR=OFF -DNETCDF_ENABLE_LIBXML2=OFF -DNETCDF_ENABLE_NCZARR_FILTERS=OFF -DNETCDF_ENABLE_V2_API=OFF -DNETCDF_ENABLE_LIB=OFF -DNETCDF_ENABLE_BASH_SCRIPT_TESTING=OFF -DNETCDF_BUILD_UTILITIES=OFF -DHDF5_DIR="$Env:INSTALL_PATH"/cmake -DHDF5_C_INCLUDE_DIR="$Env:INSTALL_PATH"/include -DHDF5_hdf5_LIBRARY_RELEASE="$Env:INSTALL_PATH"/lib/libhdf5.lib -DHDF5_hdf5_hl_LIBRARY_RELEASE="$Env:INSTALL_PATH"/lib/libhdf5_hl.lib -DZLIB_LIBRARY="$Env:INSTALL_PATH"/lib/zlibstatic.lib -DZLIB_INCLUDE_DIR="$Env:INSTALL_PATH"/include -DSzip_DEBUG_LIBRARY="$Env:INSTALL_PATH"/lib/szip-static.lib -DSzip_RELEASE_LIBRARY="$Env:INSTALL_PATH"/lib/szip-static.lib -DSzip_INCLUDE_DIRS="$Env:INSTALL_PATH"/include ..
cmake --build . --target install --config Release
cd ../../..
The issue is that NetCDF is implicitly defining H5_BUILT_AS_DYNAMIC_LIB due to the commented out code here https://github.com/Unidata/netcdf-c/blob/main/cmake/dependencies.cmake#L126
- Can that code be uncommented? As it is now, netcdf includes hdf5 incorrectly.
Once I did that https://github.com/Unidata/netcdf-c/commit/e9325d56d5d7edc392b805dbc9be3f41bb08742a, it detected hdf5-static, but I hit the following error
-- Checking for HDF5 config header
-- Looking for H5public.h
-- Looking for H5public.h - found
-- Using H5public.h to check for feature macros
-- Checking for H5_HAVE_ZLIB_H
-- Performing Test _has_H5_HAVE_ZLIB_H
-- Performing Test _has_H5_HAVE_ZLIB_H - Success
-- HDF5 has zlib.
-- Checking for H5_HAVE_FILTER_SZIP
-- Performing Test _has_H5_HAVE_FILTER_SZIP
-- Performing Test _has_H5_HAVE_FILTER_SZIP - Success
-- -- HDF5_UTF8_PATHS (HDF5 version 1.10.6+): ON
-- Looking for H5Pset_all_coll_metadata_ops
CMake Error at C:/netcdf/netcdf-c-4.9.3/build/CMakeFiles/CMakeScratch/TryCompile-537ruh/cmTC_639b9Targets.cmake:31 (set_target_properties):
The link interface of target "hdf5-static" contains:
ZLIB::ZLIB
but the target was not found. Possible reasons include:
* There is a typo in the target name.
* A find_package call is missing for an IMPORTED target.
* An ALIAS target is missing.
I believe that this is an HDF5 issue, since I also saw the same thing with CGNS https://github.com/CGNS/CGNS/pull/834. The difference is that here is that there's already a call to find_package(ZLIB).
I then saw that netcdf bundles its own FindZLIB.cmake, which doesn't define ZLIB::ZLIB, so I deleted that file from cmake/modules https://github.com/Unidata/netcdf-c/commit/e9325d56d5d7edc392b805dbc9be3f41bb08742a and upon reconfiguring, the ZLIB::ZLIB target was defined.
- Does NetCDF need its own custom version or can
FindZLIB.cmakebe removed? CMake has included support for ZLIB since version 3.1.
Finally, I had to add a call to find_package(libaec) https://github.com/Unidata/netcdf-c/commit/677e5caddd0c198ef1a5b065062d0402f98f5fe0, similar to https://github.com/CGNS/CGNS/pull/847.
With that, I was able to build and install NetCDF statically on windows, and my application linked successfully as well.
Thank you for the report, and the sample script. I'll take a look at this, and see what we need to change on our end; removing the FindZLIB.cmake we bundle, and requiring the cmake version be 3.1+ shouldn't be a problem. I'm setting up to try to replicate this and then I'll make the changes to address this issue. Thanks!
Thank you for your patience!
I built the cmake/modules/FindZLIB.cmake and several other compression libraries. This is quite a while ago, so it is possible that some of these FindXXX.cmake files can now be deleted.
