cargo-mutants
cargo-mutants copied to clipboard
Separately & optionally run doctests
Follow-on from #223, #85:
nextest doesn't run doctests, but we could separately run cargo test --doc afterwards. In some trees, doctests might give important coverage -- although presumably not in trees that expect or require nextest.
Doctests are relatively slow compared to other Rust tests, so, perhaps people would not always want to run them. On the other hand, if mutants are caught by other tests we don't need to run the doctests, and if there are no doctests then it's fast. So, we'd only pay the performance price in cases where doctests exist, and other tests didn't catch the mutant, and in that case probably it does make sense to run them.
So perhaps this should be a true/false/auto ternary, where unspecified or auto means "true for cargo test, false for nextest." Then you could also turn them off for cargo test if you have doctests but only as testable documentation, and you don't want to rely on them.
(This requires a little refactor to introduce the concept of running more than one command per phase.)