cargo-benchcmp

cargo-benchcmp copied to clipboard

It would be great if it was possible to filter such that only statistically significant changes were shown. This is somewhat similar to issue #4.

sicking

Windows pipes in UTF-16

1

One windows, piping the benchmarks like `cargo bench > old` will save the results in UTF-16. Running `cargo benchcmp` will then fail with "stream did not contain valid UTF-8".

cbreeden

Pick the best of multiple runs

4

One relatively simple way to paper over variability of benchmarks (for example cpu warmup-related things) is to pick the best time of multiple runs. The tool could allow having multiple...

bluss

Thanks for this utility! I think it would be nice to indicate differences inside the control error somehow. Perhaps colour them **orange/warning-colour** instead of red when a subsequent bench is...

alexheretic

use more rigourous statistics

2

Russ Cox has written https://godoc.org/rsc.io/benchstat which does the same thing but with better math.

dgryski

Set up test coverage badge

5

I saw [another crate](https://github.com/phsym/prettytable-rs) use a badge for test coverage using a service called [Coveralls](https://coveralls.io). I have no experience with them, but it looks nice. Might be a good idea...

Apanatshka

N-way comparison with plots

3

This is the big feature I've been working on. I would link you to the branch, but my modem is broken and I forgot to bring the code/laptop to work....

Apanatshka

atouchet

This will now parse, but silently truncate the new fractional `libtest` output. Fixes #48

Swatinem

https://github.com/rust-lang/rust/pull/124774 added support for outputting fractional `ns` values, which are not parsed properly (yet).

Swatinem