ghz icon indicating copy to clipboard operation
ghz copied to clipboard

Add thresholds

Open bojand opened this issue 5 years ago • 1 comments

It would be useful to have a threshold settings within project options for different statistical metrics (ie fastest, slowest, average, percentiles) so we can report which ones fail the threshold. Additionally we could have a "key metric" setting that would dictate if a test run / report fails based on the threshold setting for that metric (in addition to errors). So if the key metric fails the threshold even if no errors, the test run / report would be considered as a fail status.

This would involve changes to database and schemas.

We could graph thresholds along with the metrics in graphs. For example in change over time we could have the thresholds (or at least the "key metric" threshold) on the change over time. Additionally we could have it marked in histogram perhaps and the comparison charts.

bojand avatar Jan 26 '19 23:01 bojand

+1 this feature. Using thresholds within a test to pass/fail the report would be extremely useful information for performance gating. I might recommend k6's implementation of this as a reference point. https://docs.k6.io/docs/thresholds. This would be useful at the HTML report level, as well as the web.

minddazed avatar Jun 05 '19 18:06 minddazed