github-action-benchmark icon indicating copy to clipboard operation
github-action-benchmark copied to clipboard

Cargo-criterion tool support

Open brainstorm opened this issue 2 years ago • 6 comments
trafficstars

After struggling with cargo bench integration with criterion-rs and this action, I've decided to add support for cargo criterion which adds improvements in several areas vs the plain criterion-rs bench+plaintext extract approach.

Here are some examples of the outputs generated by this PR:

https://umccr.github.io/htsget-rs/dev/bench/ <--- generated by this action https://umccr.github.io/htsget-rs/dev/bench/native/criterion/reports/ <--- generated by cargo-criterion natively and pushed to gh-pages branch as-is.

/cc @mmalenic

brainstorm avatar Dec 22 '22 05:12 brainstorm

When cargo criterion output benchmark results for multiple functions, it seems that only one of them is included? Is it possible to parse all benchmark results from the output of cargo criterion?

dclong avatar Feb 15 '23 07:02 dclong

When cargo criterion output benchmark results for multiple functions, it seems that only one of them is included? Is it possible to parse all benchmark results from the output of cargo criterion?

Oh, sorry about that, can you point me in the code or show me some errors where you are having issues? IIRC it works for our multiple cargo criterion benchmark outputs, so I need a bit more details to diagnose this issue...

brainstorm avatar Feb 16 '23 02:02 brainstorm

When cargo criterion output benchmark results for multiple functions, it seems that only one of them is included? Is it possible to parse all benchmark results from the output of cargo criterion?

I'm still getting the same issue here. You can have a look at my workflow execution here: https://github.com/GitGab19/stratum/actions/runs/5795828071/job/15708142675 And here what it's generated from this action: https://gitgab19.github.io/stratum/dev/bench/

GitGab19 avatar Aug 08 '23 11:08 GitGab19

If anyone else is blocked by this, Bencher supports Criterion: https://github.com/bencherdev/bencher#supported-benchmark-harnesses

epompeii avatar Aug 16 '23 12:08 epompeii