How to detect presence and capabilities of z5 from autoconf/CMake build systems?

[edwardhartnett]

Howdy!

I am one of the programmers on the PIO project, a free software C/Fortran library to do scalable, netCDF I/O on many many processors on modern supercomputers. (See https://github.com/NCAR/ParallelIO).

The PIO library is used by the CESM climate model (http://www.cesm.ucar.edu/). A CESM programmer contributed changes to the PIO library that uses z5 to add Zarr capability, and they are very happy with the results.

I am trying to merge their work, and I need to understand how to detect whether Z5 is present on the build machine. I need to do this from both autoconf and CMake based builds.

Also I need to learn which compression filters are available with Z5. Is there a good way to do that?

Since z5 seems to be a header only library, I guess I can't look for a function, which is what I would usually do...

[constantinpape]

The PIO library is used by the CESM climate model (http://www.cesm.ucar.edu/). A CESM programmer contributed changes to the PIO library that uses z5 to add Zarr capability, and they are very happy with the results.

Very glad to hear this :).

I am trying to merge their work, and I need to understand how to detect whether Z5 is present on the build machine. I need to do this from both autoconf and CMake based builds.

Indeed, z5 is header only (and compression libs need to be linked dynamically). Unfortunately, I haven't come around to implement a canonical way to check for it in CMake yet. I think the proper way to do this would be to add a z5Config.cmake, so that find_packages(z5) is available once z5 has been installed. See also https://cmake.org/cmake/help/v3.0/manual/cmake-packages.7.html#creating-packages.

Would that work for you / any other suggestions? If you think this would meet your requirements, a PR which adds this would be very welcome :). (Otherwise, I can look into it after Wednesday the 18th, am very busy till then.)

Also I need to learn which compression filters are available with Z5. Is there a good way to do that?

I would also do this via find_packages, e.g. for blosc you could do something like this:

find_packages(BLOSC)
if(BLOSC_FOUND)
  # build with blosc ....
endif()

See also https://github.com/constantinpape/z5/blob/master/CMakeLists.txt#L218. Would this work for you?

Let me know if you run into any other issues or have suggestions to improve the build process.

[edwardhartnett]

Thanks for the suggestions. I will look into this and get back to you with whatever I come up with...

[constantinpape]

FYI I started to work on this in #168

[edwardhartnett]

OK, but actually I also need this in autoconf. ;-)

What autoconf needs is a header file to look for, and a library to link to. Since you don't actually have a library, but just header files, I'm not sure how to proceed. I guess a custom test which attempts to build a program that uses z5.

On Sat, Mar 28, 2020 at 2:06 PM Constantin Pape [email protected] wrote:

FYI I started to work on this in #168 https://github.com/constantinpape/z5/pull/168

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/constantinpape/z5/issues/167#issuecomment-605512381, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJIOMMGY2ZADGJG4DTTCDS3RJZKDNANCNFSM4LF4672A .

[constantinpape]

What autoconf needs is a header file to look for, and a library to link to. Since you don't actually have a library, but just header files, I'm not sure how to proceed. I guess a custom test which attempts to build a program that uses z5.

For the header, I could just add a z5.hxx header. About the library I am not sure; I don't have any experience with autoconf.

[constantinpape]

Ok, in #168 I have added a cmake config now and tested it in one of my external projects and it seems to work. I have also added a z5.hxx header. @edwardhartnett let me know if you need anything else to make it work with autoconf.