cargo-aoc
cargo-aoc copied to clipboard
Allow overriding criterion configs
On Advent of Code 2020, Day 15, part 2. I'm trying to benchmark things in the 1s range in performance
runner: 1.159830989s
With the default criterion config that produces an estimate of:
Benchmarking Day15 - Part2/alt: Collecting 100 samples in estimated 6081.4 s (5050 iterations)
This is quite a long time to wait. Ideally, we would be able to annotate the individual test to override the default criterion config.
I'm able to work around this issue by editing target/aoc/aoc-autobench/benches/aoc_benchmark.rs
and changing:
criterion_group!(benches, aoc_benchmark);
into
criterion_group! {
name = benches;
config = Criterion::default().significance_level(0.05).sample_size(15);
targets = aoc_benchmark
}