HDFGroup
On macOS, using CMake, the OS-provided zlib is not found. Worse, it seems not possible to point to it.
Describe the bug On macOS, using CMake, the OS-provided zlib is not found. Worse, it seems not possible to point to it.
HDF5 provides a ZLIB_DIR setting, but it's described as "The directory containing a CMake configuration file for ZLIB." macOS does not provide any CMake configuration file for ZLIB, and in fact CMake is not even zlib's usual build system, so it's odd to look for that at all.
Other libraries I use, such as VTK and ITK, provide CMake variables like:
ZLIB_INCLUDE_DIRZLIB_LIBRARY_DEBUGZLIB_LIBRARY_RELEASE
Which you can just point directly to the zlib include directory, and library files. This is most flexible, and then you don't care how zlib was built.
These tickets seem related, but maybe not identical: https://github.com/HDFGroup/hdf5/issues/4614 https://github.com/HDFGroup/hdf5/issues/4904
Expected behavior macOS provides zlib, therefore I'd expect it to be detected by default, or at least be able to point to it.
Platform (please complete the following information)
- HDF5 version: git master
- OS and version: macOS 14
- Compiler and version: Xcode 16
- Build system (e.g. CMake, Autotools) and version: CMake
Hi, @seanm !
Thank you for your report but I cannot duplicate your concern.
-- Found ZLIB: /Applications/Xcode_16.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.1.sdk/usr/lib/libz.tbd (found version "1.2.12")
-- H5_ZLIB_HEADER=zlib.h
-- H5_ZLIB_HEADER=zlib.h
-- H5_ZLIB_HEADER=zlib.h
-- H5_ZLIB_HEADER=zlib.h
-- H5_ZLIB_HEADER=zlib.h
Please see the complete action log and workflow.
All (incl. deflate) tests passed.
@hyoklee thanks for your reply. In my case, it finds a zlib that some other software installed. It finds ZLIB_DIR as /usr/local/minc/share/cmake/ZLIB. That's not so bad, but I can't find a way to point it elsewhere. What do I set ZLIB_DIR to point to the macOS-provided zlib?
In release_docs/INSTALL_CMake.txt , section "IV. Further considerations" and section "V. Options for Building HDF5 Libraries with the CMake Command Line" show the CMake options to use.
* H5_SZIP_INCLUDE_DIR:PATH=<path to szip includes directory>
* H5_SZIP_LIBRARY:FILEPATH=<path to szip/library file>
* H5_ZLIB_INCLUDE_DIR:PATH=<path to zlib includes directory>
* H5_ZLIB_LIBRARY:FILEPATH=<path to zlib/library file>
We may have been overzealous with our prefix (H5_) changes and you may need to remove the prefix.
This is not macOS specific -- the same problem occurs on Windows if using a self-compiled ZLIB. ZLIB itself does not provide a cMake configuration file when installed so that isn't a Mac thing, it's just a ZLIB thing. It does provide a pkgconf configuration file, but HDF5 is not attempting to use that. To get it working I had to manually set the paths to the include dir and the direct path to the library file itself, via: `-D ZLIB_INCLUDE_DIR={self.install_dir}/include -D ZLIB_LIBRARY={self.install_dir}/lib/zlib.lib" (note that the H5 prefix specified in the documentation is indeed incorrect, and should not be included).
In release_docs/INSTALL_CMake.txt , section "IV. Further considerations" and section "V. Options for Building HDF5 Libraries with the CMake Command Line" show the CMake options to use.
* H5_SZIP_INCLUDE_DIR:PATH=<path to szip includes directory> * H5_SZIP_LIBRARY:FILEPATH=<path to szip/library file> * H5_ZLIB_INCLUDE_DIR:PATH=<path to zlib includes directory> * H5_ZLIB_LIBRARY:FILEPATH=<path to zlib/library file>
I see no such options in cmake:
I think those vars are for the find_library process. There is a mention of this: You can specify the search order using one or more of NO_DEFAULT_PATH, NO_CMAKE_ENVIRONMENT_PATH , NO_CMAKE_PATH, NO_SYSTEM_ENVIRONMENT_PATH, NO_CMAKE_SYSTEM_PATH, CMAKE_FIND_ROOT_PATH_BOTH, ONLY_CMAKE_FIND_ROOT_PATH, or NO_CMAKE_FIND_ROOT_PATH.
CMAKE will search under $CMAKE_PREFIX_PATH/include for headers and $CMAKE_PREFIX_PATH/libs for libraries.
From CMAKE documentation: For each path in the CMAKE_PREFIX_PATH list, CMake will check "PATH/include" and "PATH" when FIND_PATH() is called, "PATH/bin" and "PATH" when FIND_PROGRAM() is called, and "PATH/lib and "PATH" when FIND_LIBRARY() is called.
These tickets seem related, but maybe not identical: https://github.com/HDFGroup/hdf5/issues/4614 https://github.com/HDFGroup/hdf5/issues/4904
This is identical to #4904.
HDF5 should stop doing a non-standard search of explicitly looking for config file first, then falling back to builtin FindZLIB.cmake. Just do the builtin find_package(ZLIB) / FindZLIB.cmake instead, given how rare zlib with CMake config file is -- it's unlikely to be what the user wants.
Users can decide from the outside to pass -DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ON. That's not something HDF5 should decide and enforce for the user.
This is still planned for 2.0 release - I agree with the solution - just haven't had time to work on it. Concentrated on libaec issues.
This bug is pretty urgent IMHO, it's impossible to build HDF5 if you turn on HDF5_ENABLE_ZLIB_SUPPORT because of this.
I second @haampie use the module version of find_package so that the users can use the variable -DZLIB_ROOT=/path/to/zlib to specify their own ZLIB.
@byrnHDF thanks for the PR! I just tried and it seems to work for me! It finds the macOS zlib, not other random ones I have around. And it compiles, links, and all tests pass.
I do hope this makes it into the next patch release, and not only in 2.0. As I said, there seems to be no workaround otherwise!
Hi @seanm,
have you tried the 1.14.6 release just put out a few days ago? These changes were put into that release, as well as a fix/unfix for a unicode issue.
I ran 'cmake (3.24) -G "Unix Makefiles" ../hdf5-1.14.6' again on our macOS 15.3 machine. It found /usr/lib/libz.1.2.12.dylib. I don't know that we have other random zlibs around on that machine, but without that sort of complication it found the system installed zlib. The build passed all tests including 23 "deflate" tests.
have you tried the 1.14.6 release just put out a few days ago?
No, I didn't even know about that release
These changes were put into that release
They were? But the PR isn't even merged yet.
They were? But the PR isn't even merged yet.
We had some discussions around minor issues that I brought up with the PR, but the changes went into the release in https://github.com/HDFGroup/hdf5/pull/5284 before the PR to develop was merged. A bit different from the usual way of doing things, but we wanted to get the 1.14.6 release out in a timely manner to fix issues in h5py.