cargo-mutants icon indicating copy to clipboard operation
cargo-mutants copied to clipboard

RFE: generate a sourcemap of what code affects which tests

Open mathstuf opened this issue 1 year ago • 4 comments

It would be nice to be able to generate some kind of database that records what tests cover which source lines. This could then be consumed by something like nextest in conjunction with git diff to only run tests that are affected by the changes in a patch to avoid running the entire test suite all the time.

mathstuf avatar Oct 29 '24 20:10 mathstuf

As a bonus, CI could also mutate just the diff and run tests in that direction too.

mathstuf avatar Oct 30 '24 02:10 mathstuf

This is similar to #24.

But, maybe here you're saying we could remember which specific tests failed due to each specific mutation?

sourcefrog avatar Oct 31 '24 18:10 sourcefrog

But, maybe here you're saying we could remember which specific tests failed due to each specific mutation?

Yes. When discussing with colleages, they suggested doing it by coverage, but I'm not aware of any coverage tools that save such information in a per-test bucket (especially in-process testing like Rust's #[test] or pytest).

mathstuf avatar Oct 31 '24 19:10 mathstuf

Yes, I think it's an interesting idea. I think it's more closely connected to a coverage tool than to cargo-mutants, and it would probably require running each test separately like Nextest does.

sourcefrog avatar Nov 04 '24 15:11 sourcefrog