CDash
CDash copied to clipboard
The threshold shown in the "Test Timing" message is not the one used for the test
Here is an example on a public CDash: https://open.cdash.org/test/651305806?graph=time, where we can read:
Test Timing: Failed This test took longer to complete (21s 990ms) than the threshold allows (26s 870ms).
But according to the graph, it seems to me that
- the threshold for the test was 20.96 s
- the "26.87 s" duration mentionned in the message is actually the updated threshold
I assume the intended behavior is to display the actual threshold (or the phrasing is very confusing!).
Note that the "acceptable range" curve (in the "Test Time" graph) should be shifted to the right as its measurement is always one day ahead. It would be better to represent its value at the time of the test and not its new calculated value to better compare it to the “Execution Time” curve.