reframe
reframe copied to clipboard
A powerful Python framework for writing and running portable regression tests and benchmarks for HPC systems.
In cscs we have (almost) separated all the systems configuration in different files for better readability but now we have to pass them all manually in the cli `-C common.py...
That would be a very nice and handy addition.
PR #2833 adds the ability to configure the generated child pipelines with arbitrary options using test variable `ci_extras`. Currently, this variable is defined in the Regression test class and defaulted...
It would sometimes be useful to have an attribute that you can set in the init phase of a regression test which will either exclude or include files from be...
This will make much easier the navigation inside the stage, output and perflog dirs. Note that the hashed named will be used internally, as the special characters of the human...
I am currently doing some experiments where I generate many system partitions on the fly, based on a common topology. E.g `system:gpu1`, `system:gpu2`, `system:gpu3` which are more or less copies...
Some log handlers (e.g. `httpjson`) do not ship a consistent set of unit tests. This should be addressed on the medium term.
Currently the thresholds are essentially percentages relative to the reference value. As a result, reference values cannot be 0, as also the bounds will be set to zero and if...
The logging mechanism of 4.x introduced new mechanics for performance tests where a test log is produced at the completion of each test, with large improvement within the handlers ....
More information here: https://docs.readthedocs.io/en/stable/pull-requests.html#preview-documentation-from-pull-requests All we need to do is to set `Build pull requests for this project` in the advanced settings of the rtd project: https://docs.readthedocs.io/en/stable/guides/pull-requests.html