cargo-mutants
cargo-mutants copied to clipboard
Time estimation shouldn't include initial build and baseline tests
Today, the extrapolation of completion time includes the time for the initial clean build (if not in-place) and for the baseline tests. This helps compute a worst-case estimate, but it's often very pessimistic, for two reasons:
- The incremental builds will typically be much faster than the baseline build
- Many tests of mutants are likely to fail fast and therefore be much faster than the baseline tests.
Probably it would be better to measure time, for estimation of ETA, only after the baseline tests have succeeded.
(This will get a little inaccurate with multiple jobs, because some of them will need to do clean builds, but measuring from after the baseline tests will still be an improvements.)
Thank you @panarch!