zlib icon indicating copy to clipboard operation
zlib copied to clipboard

CMake: Make tests optional

Open robiwano opened this issue 7 years ago • 16 comments

Enclose the example executables example/example64 with

option(ZLIB_ENABLE_TESTS "Set to ON to enable the ZLIB tests" OFF)
if (ZLIB_ENABLE_TESTS)
... add tests here ...
endif ()

otherwise those tests are included in projects that use zlib via submodule and CMake.

robiwano avatar Oct 24 '17 11:10 robiwano

One way to make sure tests are built when building zlib standalone:

if (${CMAKE_PROJECT_NAME} STREQUAL zlib)
  option(ZLIB_ENABLE_TESTS "Set to ON to enable the ZLIB tests" ON)
else ()
  option(ZLIB_ENABLE_TESTS "Set to ON to enable the ZLIB tests" OFF)
endif ()

robiwano avatar Oct 24 '17 11:10 robiwano

It takes an extra second or two for the tests to link and run. Is this worth the effort? When you run cmake --build . --target install you don't get any of the test executables installed. What's the use case?

jeking3 avatar Nov 23 '17 02:11 jeking3

One case is that projects using the zlib as submodule (via CMakeLists.txt) might have the same target names, and CMake does not allow duplicate target names. "example" is VERY generic. If they aren't excluded, at least rename the test targets as zlib_example, zlib_example64 ?

robiwano avatar Nov 23 '17 07:11 robiwano

That is quite reasonable, thanks for the addition info. If zlib cmake was in a cmake project() would that properly scope it?

jeking3 avatar Nov 23 '17 14:11 jeking3

Not sure I understand what you mean by that, but if you mean that a top level project including zlib with add_subdirectory(path/to/zlib EXCLUDE_FROM_ALL), it would work nicely.

robiwano avatar Nov 23 '17 14:11 robiwano

According to the CMake docs, it's best to use a "include(CTest)" and put tests inside an "if(BUILD_TESTING)" block

henryborchers avatar Jun 13 '18 19:06 henryborchers

I absolutely can't believe this issue is not fixed yet. I have included zlib as submodule and the example test is failing (for whatever reason). And there is no way to disable it. What a shame.

aneeshpb-amazon avatar Jun 23 '23 12:06 aneeshpb-amazon

I'm also pretty flabbergasted that this is still a problem 6 years after the issue was raised. I'm using zlib as a git submodule in an embedded project with zero support for executables. And it is seriously annoying to have to manually comment out the section trying to build the executables to make it build and then have to revert it again to be able to submit changes to the parent module.

kavionic avatar Jun 28 '23 21:06 kavionic

7 years... still no fix?

cloudwindy avatar Jan 25 '24 08:01 cloudwindy

@madler: Can you look it?

Neustradamus avatar Jan 25 '24 16:01 Neustradamus

I guess you meant madler :see_no_evil: , right?

rgpublic avatar Jan 25 '24 16:01 rgpublic

How am I supposed to know what the right thing to do is? It seems like every time I apply a patch to fix someone's cmake problem, someone else's cmake workflow gets screwed up.

madler avatar Jan 25 '24 16:01 madler

I have included zlib as submodule and the example test is failing (for whatever reason).

The "whatever reason" may be important. example is there in the first place to test for basic problems with the zlib build. How exactly is it failing?

This leads me to think that the example test should not in fact be made optional, since there should be at least a minimal test that the zlib build is correct.

madler avatar Jan 25 '24 16:01 madler

I have included zlib as submodule and the example test is failing (for whatever reason).

The "whatever reason" may be important. example is there in the first place to test for basic problems with the zlib build. How exactly is it failing?

This leads me to think that the example test should not in fact be made optional, since there should be at least a minimal test that the zlib build is correct.

Yeah what you're saying is true. But as a consumer of zlib (via submodule), I don't want to run any zlib tests during builds. Only zlib devs should even be running these tests. But I agree that having a test fail indicates a deeper problem.

Unfortunately I don't have context anymore, since I've moved away from the project that used zlib. But it seems that a simple change could unblock a lot of people.

aneeshpb-amazon avatar Jan 25 '24 16:01 aneeshpb-amazon

https://github.com/libjxl/libjxl/actions/runs/7671453687/job/20909823847

Here's an example of a test in zlib failing for us when using msan. It is flaky.

mo271 avatar Jan 26 '24 22:01 mo271

https://github.com/libjxl/libjxl/actions/runs/7671453687/job/20909823847

Here's an example of a test in zlib failing for us when using msan. It is flaky.

That is with 1.2.11. Use the most recent version (1.3.1).

madler avatar Jan 26 '24 23:01 madler

6201f893846bfd22faf060e84a3bb7bfc0e61761 added the requested option.

madler avatar Feb 12 '24 17:02 madler