mutation-testing-elements icon indicating copy to clipboard operation
mutation-testing-elements copied to clipboard

Proposal: new mutation status for mutants that are covered by a test that fails even before mutation

Open rouke-broersma opened this issue 6 years ago • 5 comments

In stryker-net we have coverage analysis so we can determine which mutants are covered by which tests. Currently we do not allow a mutation testrun if you have failing unit tests because this would mark every mutant as killed. With coverage analysis we can disable all unit tests that are failing while also marking and skipping all mutants covered by those failing unit tests. We would like to have a new status in the schema to reflect a mutant that is covered by a test but for which we cannot determine if it's killed or not due to the covering unit test not working.

rouke-broersma avatar Oct 18 '19 12:10 rouke-broersma

@nicojs thoughts?

rouke-broersma avatar May 15 '20 09:05 rouke-broersma

Can we use the ignored status with this and reflect the reason in a (new) reason field?

nicojs avatar May 15 '20 10:05 nicojs

Yes that's a good idea, we can fill the reason field already for others already as well.

rouke-broersma avatar May 15 '20 12:05 rouke-broersma

Great, let's do that than. Btw, I would suggest to have a compiler flag to disable this feature in Stryker, since you might be fooled by the mutation score.

nicojs avatar May 16 '20 15:05 nicojs

Proposal: Ability to tag mutants with traits such as:

'Slow', 'Static context', 'Initial testrun failed' etc instead of new mutant statusses

rouke-broersma avatar Sep 28 '20 15:09 rouke-broersma