criterion.rs icon indicating copy to clipboard operation
criterion.rs copied to clipboard

Introduce criterion::Criterion::MIN_SAMPLE_SIZE associated constant

Open JohnScience opened this issue 2 years ago • 4 comments

Currently, the lower bound for criterion::Criterion<M: Measurement>::sample_size() is hard-coded as 10. I would like to run some macro-benchmarks as few times as possible, i.e. precisely 10 times. I suspect there are other people who would love to do this too.

JohnScience avatar Nov 15 '21 08:11 JohnScience

Could you elaborate on your use-case? Why exactly do you want the minimum sample size? And what statistical properties are you looking for in the results?

lemmih avatar Nov 15 '21 08:11 lemmih

I want the minimum sample size because I might need to perform measurements on large volumes of different data. I want to see how different approaches work on different data in average and how calculation times are distributed but I don't want to keep my computer (or VDS) running all the tasks. I have time before spring but want to iterate quickly. For that, even some coarse benchmarking data is sufficient.

JohnScience avatar Nov 15 '21 08:11 JohnScience

Ah, so something like #110.

lemmih avatar Nov 15 '21 08:11 lemmih

Yes, the feature mentioned in #110 would be valuable.

JohnScience avatar Nov 15 '21 08:11 JohnScience