stryker-net icon indicating copy to clipboard operation
stryker-net copied to clipboard

Ability to disable mutant, that end up with timeout

Open CzBuCHi opened this issue 10 months ago • 6 comments

Is your feature request related to a problem? Please describe.

I have this code:

List<string> list = new List<string>();
string Aggregate() { 
    var top = list[0];
    list.RemoveAt(0); 

    if (top.All(char.IsNumber)) {
        return top;
    }

    var left = Aggregate();
    var right = Aggregate();
    return "(" + left + top + right + ")";
}

when stryker removes list.RemoveAt(0); it will result in surviving mutant due to timeout ...

Describe the solution you'd like Ability to use 'timeout' keyword when disabling mutators via comments:

// Stryker disable once timeout
list.RemoveAt(0); 

Describe alternatives you've considered im currently using // Stryker disable once all but that feels wrong because some mutant may still squeeze in there ...

Additional context Same thing applies to uncovered parts of code:

public string Foo(int bar){
    if (bar == 3 || bar == 5) {
        return FooImpl(bar); // this is the only call to FooImpl
    }

    return "BAZ";
}

private string FooImpl(int bar) {
    return bar switch {
               3 => "FOO",
               5 => "BAR",
               // Stryker disable once all     <= this is a hole for mutant to sneak thru...
              // dotCover disable next line <= dotCover will not count next line to coverage
               _ => throw new UnreachableException(),
           };
}

CzBuCHi avatar Apr 10 '24 18:04 CzBuCHi

We wouldn't know it's a timeout until after testing, so how can we disable the mutant from testing?

Instead of disable all you can specify the type of mutator to disable here.

rouke-broersma avatar Apr 10 '24 20:04 rouke-broersma

i would interpret // Stryker disable once timeout as if timeout occurs after this line, do not show it in report

CzBuCHi avatar Apr 11 '24 06:04 CzBuCHi

it will result in surviving mutant due to timeout ...

This is not true, timeouts are counted as killed when calculating the mutation score. So why would you want to exclude timeouts from the report?

im currently using // Stryker disable once all but that feels wrong because some mutant may still squeeze in there ...

How would any mutant be generated here? Maybe I'm not following correctly.

richardwerkman avatar Apr 11 '24 07:04 richardwerkman

@CzBuCHi : what is exactly the problem you are trying to solve here?

dupdob avatar Apr 11 '24 07:04 dupdob

probably my own confusion ( after reading @richardwerkman answer ) - i considered timeouts as bad & need to be fixed (its not green :/)

but i think this can still be applied to uncovered parts of code:

public string Foo(int bar){
    if (bar == 3 || bar == 5) {
        return FooImpl(bar); // this is the only call to FooImpl 
    }

    return "BAZ";
}
private string FooImpl(int bar) {
    return bar switch {
               3 => "FOO",
               5 => "BAR",
              // dotCover disable next line
               _ => throw new UnreachableException(), // this will be never called, but compiler requires it ...
           };
}

because in this case stryker will complain about uncovered parts of code ... (idk right now it that one is red or yellow)

CzBuCHi avatar Apr 11 '24 07:04 CzBuCHi

Stryker comments can be used for uncovered parts of the code, so this is already possible. The only feature request I can see here is recognizing/reusing existing comments (such as dotCover here)

dupdob avatar Apr 15 '24 14:04 dupdob

We don't currently plan supporting comments from other frameworks

rouke-broersma avatar Jun 07 '24 14:06 rouke-broersma