reframe
reframe copied to clipboard
A powerful Python framework for writing and running portable regression tests and benchmarks for HPC systems.
Includes also changes from #3054, as soon it is merged the diff will be a bit smaller. In order to test it these variables need to be set: ```bash export...
On multi-gpus nodes, it is possible to control which devices an application can use by setting CUDA_VISIBLE_DEVICES. MPI codes must use a shell script. In this example: ``` srun ./visible_devices.sh...
Unfortunately, `RFM_REMOTE_DETECT=TRUE` calls `pip` via `bootstrap.sh`. I am not able to use `pip` on my computer (usual proxy stuff). @vkarak suggests to add an option to avoid the bootstrap and...
currently, perflogs, output, and stage files are saved in the prefix when specified with `--prefix`. the non-perf file logs, however, are saved in the current working dir (if `'name'` is...
Consider the following test: ```python import reframe as rfm import reframe.utility.sanity as sn @rfm.simple_test class MyTest(rfm.RunOnlyRegressionTest): valid_systems = ['*'] valid_prog_environs = ['*'] executable = "/bin/true" @sanity_function def check_exit(self): return sn.assert_eq(self.job.exitcode,...
ReFrame is currently great at detecting performance regression automatically, but for the task of comparing system configurations (e.g. driver version) it remains a manual process: * Configure the perflog output...
a list of environment variables required by reframe would be useful in the documentation in order to ensure that reframe executes properly for example `HOME` is required. If not defined...
This would make cloning much faster
If there is a compilation error, the test will fail in the build phase anyway, so currently for a compile-only test we have to add an "always-true" validation test.
Thanks to the community we have several backends implemented, but it would make the life of those who want to implement new ones much easier. We should explain the architecture...