current-bench icon indicating copy to clipboard operation
current-bench copied to clipboard

Github bot messages

Open art-w opened this issue 4 years ago • 1 comments

It's easy to forget checking the benchmarks on each PR, because it's generally not necessary.

Current-Bench could detect when a metric moves by more than 10% (?) and send a warning message to the PR: "You probably want to check those metrics!" For inspiration, see an example codecov report with pretty graphs embeded (but I think we should aim for a shorter message than this!)

Both regressions and improvements should be reported: a benchmark improvement can actually be indicative of a bug where real work has been skipped! And it's otherwise a nice confirmation that the PR fixed a performance issue :)

art-w avatar Feb 18 '22 16:02 art-w

See also #143 though this seems nicer to me.

punchagan avatar Feb 18 '22 19:02 punchagan