github-action-benchmark icon indicating copy to clipboard operation
github-action-benchmark copied to clipboard

catch2 time units are ignored when graphing and generating alerts

Open paradust7 opened this issue 2 years ago • 1 comments

When using the "catch2" extractor, the time units, "ms", "us", etc, are extracted and printed, but otherwise ignored.

This results in broken graphs when the time is near a unit boundary:

broken_graph

It also results in false alerts:

Screenshot from 2022-05-04 17-17-24

paradust7 avatar May 05 '22 00:05 paradust7

Any news on this? getRatio doesn't involve unit calculation atm, though it seems like an easy fix for a problem that makes this action pretty useless for catch: https://github.com/benchmark-action/github-action-benchmark/blob/8e749722e912d8d81d0ef090f927695c58efcf5e/src/write.ts#L567

wwerkk avatar Apr 18 '24 14:04 wwerkk