tmt icon indicating copy to clipboard operation
tmt copied to clipboard

CMake test framework plugin

Open LecrisUT opened this issue 11 months ago • 6 comments

I want to sketch out some ideas for making tmt-cmake plugin:

  • [ ] overall: how to write a tmt plugin? E.g. how can it be included in the .fmf files, how to overwrite it via an editable-like installation, how to minimize coupling with upstream
  • [ ] overall: plugin will make sure CMake is available (maybe also ninja for good measure) from package manager
  • [x] plan.prepare plugin: runs a configure-build (maybe also install) preset/traditional workflow
    • Source folder is in fmf tree, and build folder is in plan-data
    • Currently will limit that only one CMake prepare step is present
    • All other steps, must check that this step exists and use the build folder from this step (hence why only 1 CMake prepare step currently)
  • [ ] test.framework plugin: run ctest for a build environment defined in previous feature
    • Currently considering, this just runs the full ctest suite with the filters defined in plan.discover
  • [x] plan.discover plugin: convert test names and ctest labels to tmt tests and tags
    • https://cmake.org/cmake/help/latest/manual/ctest.1.html#show-as-json-object-model
    • Blocked by #2813
    • Currently considering, this step only generates one super-test and the user can use this step to add the ctest filters
  • [ ] ~~plan.report plugin: recover junit results from ctest and forward it down~~
    • Need to rethink if this is necessary
  • [ ] CMake: make tmt commands and/or python environment available in CMake
  • [ ] CMake: add test wrapper to convert tmt-native test to ctest callable tests
  • [ ] CMake: expose a tmt subcommand to be executed as part of ctest --build-and-test. E.g. thic could parse the build stdout/stderr, build folder artifacts, etc. as part of a regression test and pass it to something like beakerlib.

LecrisUT avatar Mar 12 '24 23:03 LecrisUT