criterion.rs
criterion.rs copied to clipboard
bheisler
Preliminary results / Fast mode
Hi, I really like criterion for benchmarking. Although one issue I have are the long runtimes in cases where it's clear that it's faster or slower. Currently I just use #bench upfront, which leads to a lot of code duplication. Would be nice if criterion would print preliminary results or execute #bench, to speed up code bench cycles.
Thanks for the suggestion!
Yeah, I've considered adding a quick-iteration mode, possibly based on a command line flag. It certainly would be useful.
One proposed behavior is that the benchmark suit (criterion group) is first executed for the smallest sample size (10) and then iteratively for fractions of the "recommended" sample sizes given explicitly or estimated by the criterion. The urgency of getting more precise results can be expressed as a sequence of weights (w_1,w_2,...,w_n), or, in the common case when they are equally urgent, by some special element.
Also it would be nice to have an ability to continue the computation interrupted earlier.
It seems that a fast mode is introduced in v0.4 (#535), maybe it will suffice for this feature request?
