tmt
tmt copied to clipboard
CMake test framework plugin
I want to sketch out some ideas for making tmt-cmake plugin:
- [ ] overall: how to write a tmt plugin? E.g. how can it be included in the
.fmf
files, how to overwrite it via an editable-like installation, how to minimize coupling with upstream - [ ] overall: plugin will make sure CMake is available (maybe also ninja for good measure) from package manager
- [x]
plan.prepare
plugin: runs a configure-build (maybe also install) preset/traditional workflow- Source folder is in fmf tree, and build folder is in plan-data
- Currently will limit that only one CMake prepare step is present
- All other steps, must check that this step exists and use the build folder from this step (hence why only 1 CMake prepare step currently)
- [ ]
test.framework
plugin: runctest
for a build environment defined in previous feature- Currently considering, this just runs the full ctest suite with the filters defined in
plan.discover
- Currently considering, this just runs the full ctest suite with the filters defined in
- [x]
plan.discover
plugin: convert test names andctest
labels to tmt tests and tags- https://cmake.org/cmake/help/latest/manual/ctest.1.html#show-as-json-object-model
- Blocked by #2813
- Currently considering, this step only generates one super-test and the user can use this step to add the ctest filters
- [ ] ~~
plan.report
plugin: recover junit results from ctest and forward it down~~- Need to rethink if this is necessary
- [ ] CMake: make tmt commands and/or python environment available in CMake
- [ ] CMake: add test wrapper to convert tmt-native test to ctest callable tests
- [ ] CMake: expose a tmt subcommand to be executed as part of
ctest --build-and-test
. E.g. thic could parse the build stdout/stderr, build folder artifacts, etc. as part of a regression test and pass it to something like beakerlib.