blake2_simd
blake2_simd copied to clipboard
migrate benchmarks to criterion
https://github.com/bheisler/criterion.rs compiles on rust stable, meaning you can cargo check --benches
in your CI to catch accidentally broken benches. You also get charts and statistics for free.
I've used Criterion in a few places before. The reason I haven't used it here is that it seems to take a very long time to run by default, while the test::Bencher
framework seems to give me numbers relatively quickly. At least in the context of this project, the results from test::Bencher
seem to be pretty reliable, and it's been nice to be able to do really quick benchmarks while I play with inlining flags and things like that.
Maybe you can suggest some non-default parameters for Criterion that would make it run faster? But I don't know enough about the crate to know what to try.
I guess something like this should help:
criterion_group! {
name = bench_multiprocess;
config = dont_take_an_eternity_to_run();
targets = blake2b_hash_many,
hash_sneves_blake2sp,
openssl_sha1,
}
criterion_main!(bench_multiprocess);
fn dont_take_an_eternity_to_run() -> Criterion {
Criterion::default().nresamples(1_000)
.without_plots()
.sample_size(10)
}
For more options see https://bheisler.github.io/criterion.rs/criterion/struct.Criterion.html
I'd be happy to take a pull request for this as long as we could keep the total benchmarking time under, say, 200% of what it is now.