stryker-net
stryker-net copied to clipboard
stryker-mutator
Block statement mutants should not be reported on if they are unnecessary and thus removed by the block already covered filter
Is your feature request related to a problem? Please describe. As a rule, the "Ignored" state should be used when a user ignores a mutant. That's why we're using the pinocchio emoji for it 🤥.
However, since mutant filters, the block mutants get the "ignored". This is not valid IMO. Users starting with Stryker don't understand these mutants.
Describe the solution you'd like I would expect these mutants to simply not exist at all.
Ignored The mutant wasn't tested because it is ignored. Either by user action, or for another reason. This will not count against your mutation score but will show up in reports.
https://stryker-mutator.io/docs/mutation-testing-elements/mutant-states-and-metrics/
I don't agree with either of your claims.
First of all our docs do not specify that it should be user action only so we are not in violation of our shared understanding. Secondly the inclusion or ignore status of these mutants is directly controllable by the user because the filter sets the ignored state for block mutants when that block contains other active mutants. If you ignore those mutants, the block mutant will be set to active. This is user action.
Thirdly if we don't show the existence of this mutant at all users will be confused when it suddenly shows up once the ignore other mutants. That doesn't seem valid to me either.
Lastly we also show compile and runtime error mutants, we might as well not show those either if we want to start hiding valid mutants from users.
I just got 6 participants in training who all asked what "Ignored" means, why it is there, and why the top bar isn't green while on 100%. Note that this happens with the default Stryker config (so 'no config').
Showing the mutants in this way isn't helpful.
I just got 6 participants in training who all asked what "Ignored" means, why it is there, and why the top bar isn't green while on 100%. Note that this happens with the default Stryker config (so 'no config').
Showing the mutants in this way isn't helpful.
New users will have questions, we should provide the answers. That doesn't make what we're doing incorrect. It could be, but it doesn't have to be. We could for example extend the docs with examples for the Ignored status. Could also be that we need a new status/way to show 'potential' mutants. We can discuss this some time.
You say it's not useful, I disagree. For example: It provides a user with the information (provided that we tell them in a way they understand) that if they decide to manually ignore all mutants in a block they will still have a basic coverage through the block mutator. I consider that useful.
A compromise for topbar clarity could be a tooltip or parenthesised number of mutants that were user ignored relative to the overall number of mutants ignored. This would preserve @rouke-broersma's concerns, while simultaneously giving some insight into what is happening. Just an idea, if anyone has a better proposal I'm all ears
@jensk-dev We don't currently have a way to make that distinction in the html report because conceptually the difference doesn't exist in stryker terminology. We first need to revisit and agree on definitions before we can extend them and come up with a solution. Are you joining hacktoberfest? We should take some time to discuss this topic tomorrow. @richardwerkman