Improve mutation operator utility by making harder-to-kill mutants

[bkushigian]

Some mutants are "dominated" by other higher utility mutants (that are not currently generated)

Consider a < b. Mutating this to a <= b is good: it enforces the test suite to check behavior when a == b; both programs behave identically otherwise. This is a hard to kill mutant.

Next, consider a >= b. I consider this an unproductive mutant as it is very easy to kill; a < b always differs from a >= b, and it is likely to be a very easy-to-detect mutant.

Finally, consider a != b. This is much better than a >= b because it's behavior is identical to a < b when a < b and a > b...these only differ when a is equal to b. So to detect this mutant we are forcing a test suite exercise a state where a equals b.

Detecting a != b implies detecting a >= b, but not vice versa. This means that a != b dominates a >= b; it has less utility in eliciting strong tests.

Describe the solution you'd like

For relational operators I'd recommend the following mutants (from Just et al 2012):

original	m1	m2	m3
<=	<	==	true
>=	>	==	true
==	>=	<=	false
<	<=	!=	false
>	>=	!=	false
!=	>=	<=	true

Each of these mutants is as close to the original program as possible without being equivalent.

Also, note that for == and != we are assuming the operands are orderable. If they are not, then simply mutating == to false and != to true will be the highest impact mutants. Why? Because when we check for a == b, if we sample a and b randomly they will usually not be equal (unless we are working with booleans). This means that changing it to false will usually evaluate to the same thing as a == b, making it harder to detect.

Similarly with a != b being mutated to true.

For more information on mutant domination, check out this paper by Kurtz et al, 2017

[odinvanderlinden]

First of all thank you for opening this issue. This feature request is something we have been thinking about implementing in Stryker. We envision a new "mutation level" setting in each of the Stryker frameworks. With a mutation level, one can configure the resolution of the mutations made by the different mutators. For example, mutation level 1 would optimize performance, while mutation level 3 optimizes for resolution. As this is a pretty big feature this might take some time. We are also not sure when we will be implementing this yet.