reframe
reframe copied to clipboard
Bug in tutorial with references
In https://reframe-hpc.readthedocs.io/en/stable/tutorial.html#adding-performance-references there's a definition for myhost:baseline. But as I've discovered after testing, it is more like mysystem:mypartition.
This becomes more confusing since later in the tutorial (see https://reframe-hpc.readthedocs.io/en/stable/tutorial.html#id11) a baseline.py configuration is shown where a hostname myhost is defined for the mytutorialsys system and an environment with name baseline. The correct reference for that would be mytutorialsys:default.
There are three related issues overall with the presentation:
- The issue I've described above; it is
mysystem:mypartitionfor references and needs to be fixed in the tutorial. - The order of presentation should be reversed; we should first discuss configuration files and then references.
- The
valid_prog_environsis not clearly explained. It took me a while to figure out how to request two features, (withvalid_prog_environs = ['+A +B'].) In retrospect, it is explained in https://reframe-hpc.readthedocs.io/en/stable/regression_test_api.html#reframe.core.pipeline.RegressionTest.valid_systems but it would be nice if the example showed more than just one feature.
If you agree with these issues I can fix them.