MIRTK icon indicating copy to clipboard operation
MIRTK copied to clipboard

Download and build zlib when not available

Open schuhschuh opened this issue 9 years ago • 13 comments

The zlib code is in a GitHub repository with CMakeLists.txt files. Thus, it should be easy to optionally download the files and build them as part of MIRTK when not available on the system. Further, do not include zlib.h in any MIRTK header file, only use it internally. This is currently not the case for mirtkCifstream.h and mirtkCofstream.h.

schuhschuh avatar Jan 28 '16 11:01 schuhschuh

Does BASIS have a mechanism to interface with CMake's ExternalProject ? That's typically what it would be useful for instead of vendoring yet another library under ThirdParty. Plus zlib is a library that has had security vulnerabilities in the past, so I would not feel comfortable maintaining an embedded copy under our source tree.

ghisvail avatar Feb 02 '16 17:02 ghisvail

Not yet. But I don't see what's the difference to having it downloaded via git submodule.

schuhschuh avatar Feb 03 '16 10:02 schuhschuh

Since BASIS does not support ExternalProject, I guess there is not much to discuss about anyway.

ghisvail avatar Feb 03 '16 12:02 ghisvail

This issue feels more in the SuperBuild territory when I think about it.

ghisvail avatar Feb 03 '16 12:02 ghisvail

Well, it's not that I would not like to add it. It has always been on the TODO list. There is some basics in there, but not complete. Also, ExternalProject has (or at least had) the downside that you have to have a complete super-build. Because all, including your own project must be an ExternalProject. But there are options such as Hunter.

Related CMake BASIS issue: https://github.com/schuhschuh/cmake-basis/issues/528

schuhschuh avatar Feb 03 '16 12:02 schuhschuh

ExternalProject has (or at least had) the downside that you have to have a complete super-build.

What do you mean? I have seen ExternalProject used in many projects for using gtest for instance and I don't reckon their build system being any more complex than a normal project.

Anyway, the fastest way forward is submodules, so this should be the way forward.

ghisvail avatar Feb 03 '16 12:02 ghisvail

What do you mean?

For example in case of VTK, you (or find_package respectively) rely on the existence of the VTKConfig.cmake file. This file only exists after the build (not configuration) of the ExternalProject. Hence, your find_package(VTK) within the CMakeLists.txt of your project will fail before the actual download and configuration of the external VTK project. But these steps are not performed before the actual build step.

schuhschuh avatar Feb 03 '16 12:02 schuhschuh

I can't add much more to this. I saw it done for gtest and thought it could be applicable to zlib too. But hey, you probably know better than I do regarding the CMake specifics.

ghisvail avatar Feb 03 '16 13:02 ghisvail

But these steps are not performed before the actual build step.

That's why I believe that Hunter has solved this by performing the download, configuration, and build of external dependencies already during the CMake project configuration. They don't actually use ExternalProject_Add. But if that is correct (looks like, see links to Hunter sources below), I am not a big fan of it. When you change the project configuration a number of times (switching modules or WITH options on and off), and your final build configuration would be with certain dependencies then actually not needed, you would have wasted your time b/c the dependency was built just because you switched something ON temporarily with "configure" in between. It also would slow down the CMake configure step quite some.

hunter_add_package includes the hunter.cmake file of the external project (e.g., ZLIB, which calls hunter_download which performs the external project download, configuration, and installation steps using execute_process at CMake project configuration time.

schuhschuh avatar Feb 03 '16 13:02 schuhschuh

Your call.

ghisvail avatar Feb 03 '16 13:02 ghisvail

For the purpose of the ThirdParty libs in MIRTK, I think it is sufficient to simply perform the download step by running "git clone" or CMake's file(DOWNLOAD) command during CMake configure step. Then, the ThirdParty libs can be configured as part of the project just as if the source files were included in the project's source distribution. No actual need for a more complex super-build. Such super-build approach I would implement separately which then also includes all other (more complex) dependencies such as VTK. Boost, Eigen, NiftiCLib, and ZLIB are yet rather basic dependencies to build along with the rest.

schuhschuh avatar Feb 03 '16 13:02 schuhschuh

I saw it done for gtest and thought it could be applicable to zlib too.

Of course you are right and it is possible as shown in this link if you can (and want to) set all the CMake variables otherwise set by Find<Pkg>.cmake or <Pkg>Config.cmake yourself in your own project. That means you cannot rely on the external project to set these variables (e.g., VTK_LIBRARIES, no availability of IMPORTED targets) for you and you have to maintain it yourself. Then you don't need (and cannot use) a find_package call and your fine. But then you have to have an alternative path in case you don't want to perform a super-build, but actually discover a system installation.

P.S.: That SO question is by the way from Andrew who was looking into this also in regards to supporting external project builds within BASIS.

schuhschuh avatar Feb 03 '16 13:02 schuhschuh

  • [ ] Document in ZLIB build instructions for Windows that the ZLIB library from the GnuWin32 project cannot be used! This is a prominent hit when searching the internet for a pre-built library.

schuhschuh avatar Mar 16 '16 11:03 schuhschuh