num-bigint
num-bigint copied to clipboard
use criterion.rs instead of the default bencher
The results of the default bencher are quite unstable(in successive benchmarks run without any other open programs, there were 10 % differences in the same test when comparing with benchcmp
). In case you are interested, I would be glad to rewrite the current benches to use criterion and test if there are any improvement.
I'm open to criterion. The only caveat is that dev-deps can't be optional, which means we'd only be able to test num-bigint
according to their minimum version, currently rust 1.26. We can still do build-checks for older versions though, which is probably good enough.
Another option is to move the benchmarks to a sub-crate in the repo, which we'd never publish.
I am somewhat in favor of using a sub crate, I will start implementing this in a private repo for now.
I've converted the benchmarks to criterion: https://github.com/Lemmih/num-criterion. Any interest in moving this under the rust-num
umbrella?
@Lemmih I don't really want to get in the business of comparing with other implementations. Don't get me wrong, it's useful data to have, but I think that's better done on neutral ground.
Without the shootout, would you be interested in moving it to rust-num
?
I'm open to a sub-crate in this repo as mentioned. I fear a separate repo will get neglected, and I'd rather have it live close to the code that it is meant to be testing.