catchorg
catch_discover_tests should pass labels to ctest
Description
I am using catch2/3.1.0 from conan. Currently, catch_discover_tests ignores any labels on tests. Consider this test file, with two labels.
#include "catch2/catch_test_macros.hpp"
TEST_CASE("Test Foo", "[foo]")
{
REQUIRE(true);
}
TEST_CASE("Test bar", "[bar]")
{
REQUIRE(true);
}
When I run catch_discover_tests(main), catch generates a catchtest_tests-b12d07c.cmake, which passes labels to ctest.
add_test( [==[Test Foo]==] C:/Source/catchtest/cmake-build-debug/catchtest.exe [==[Test Foo]==] )
set_tests_properties( [==[Test Foo]==] PROPERTIES WORKING_DIRECTORY C:/Source/catchtest/cmake-build-debug)
add_test( [==[Test bar]==] C:/Source/catchtest/cmake-build-debug/catchtest.exe [==[Test bar]==] )
set_tests_properties( [==[Test bar]==] PROPERTIES WORKING_DIRECTORY C:/Source/catchtest/cmake-build-debug)
set( catchtest_TESTS [==[Test Foo]==] [==[Test bar]==])
Expected: It generates set_tests_properties( [==[Test Foo]==] PROPERTIES ... LABELS [==[foo]==])`.
We have an internal modified version of catch that adds a --list-tests-as-json to the catch runner, which is then read by a modified CatchAddTests.cmake to emit labels in addition. Because this is JSON, I don't have to worry about quoting test or label names. We use https://github.com/nlohmann/json for emitting json. If we were to upstream our changes, would you be willing to add a third party dependency to catch? The alternative is to emit json with C++, which isn't as bad as it sounds.
Adding tags as labels: yes. But there have been a couple of attempts already, running into various issues, see #2383, #1590, #1600 possibly others. So a PR that does this would need to handle the issues that came up before.
Output: --list-tests-as-json is not getting in, period. What can get in is a reasonable JSON reporter, and because test/tag/reporter listings are routed through active reporter, it can provide test listing in json format.
I would like to give this a try, according to your comment in https://github.com/catchorg/Catch2/pull/2383#issuecomment-1077730621, you want the implementation to use --list-tests, parse the result, and provide both test cases and test lables at the same time, is it correct?
