reframe
reframe copied to clipboard
[enhancement] Various enhancements to the results DB feature
More specifically, this PR adds the following enhancements:
- Support for computed virtual test attributes. This mechanism allows us to extend the test queries in existing databases without having to migrate the DBs and update the test records.
basename: the test's name without any parameters (essentially the test class name). This is useful for parameter-based aggregations.sysenv: the formatted system, partition, environment combinationpdiff: the performance difference when doing comparisons
- Support for completely custom groupings. The syntax os the
CMPSPECis extended to get an explicit list of attributes for grouping test cases. - Support for arbitrary table column selection. The syntax of the
CMPSPECis extended to get an explicit list of attribute columns to show. As a result the--table-hide-columnsoption is removed as now the exact columns of the output can be specified. - Extend
--list-stored-testcasesto show the performance summary of the selected cases. It uses the sameCMPSPECsyntax. - Format parameters when storing a report/session using the parameter's format function.
- Show A/B absolute values for non-aggregated attributes/columns.
- Enable
-Efor testcase filtering. - Session extras keys are now stored unchanged in the DB (previously, they were prefixed with
$). This allows them to be properly filtered with the session filtering expressions ~~passed to--session-filter~~. - The
--session-extrasoption can now be specified multiple times. - Replace
prettytable format withoutlineand add a newgridformat. This is to allow nice printing of multiline cells. - Remove
--session-filteroption and replace it with special syntax in theCMPSPEC. This allows us to select and compare different sets of sessions. - Clean up and homogenize the internal storage API.
- All options dealing with sessions (e.g.,
--list-stored-sessions,--delete-sessions) are now accepting the extended query syntax (over time period, session uuid or session filter). - Options that used to manipulate a single session have now their names converted to plural.
Docs and unit tests are updated accordingly.