Include covering test name(s) in report

[cbrunnkvist]

Currently when somebody reviews the report, looking for a good place to start working from, they can only see source locations of the surviving mutants, whether they have any coverage, but not the test names/-suites/-files that **did provide ** the relevant coverage. Since we already have that knowledge at report time, it would be very helpful for the test developer to know where to start looking immediately, without having to go to (or generate) a separate coverage report themselfes.

The effort of having to do this before being able to contribute better test code grows both together with the size of the project and with the amount of time it takes to complete a coverage report. But even for smaller & faster projects, having that information visible in the report would definitely lower the barrier to get started.

Then there is also the analytics motivation: once we start keeping historical records of our mutation scores, we will want to know the names of the statistically most troublesome test and so on...

(I think this was raised in the gitter chat a long time ago and I think the outcome was aha ok the schema of the reports data must change together with the dashboard website...)

[hugo-vrijswijk]

Thanks for opening this issue! I fully agree, test information is a really nice feature that can give many useful insights to a report! The good news is that this feature is a lot closer now than it was a while ago. The HTML report now supports test data, showing what tests cover a mutant and what it did with that mutant. Stryker-js already implements this new report, and the Stryker dashboard can also show this information: https://dashboard.stryker-mutator.io/reports/github.com/stryker-mutator/stryker-js/master?module=instrumenter.

On the Stryker4s side, the sbt testrunner has all test information, it just doesn't collect it and pass it back to the main test-process at the moment. So, that would have to be done, and adding it to the Metrics output that will pass it to the HTML reporter.

One extra thing to think about is that at the moment, the sbt testrunner stops at the first test that fails (for better performance). That means the killedBy would only be filled by one test, instead of all tests that could kill that mutant. Do you think it would be useful to have something like a fail-fast = false config option?

[pathanac]

@hugo-vrijswijk when you mention "lot closer", is it something in a near future? for fail-fast=false personally, it might be useful in the future (when we want to optimize test case number) but most likely not using it now for our current use cases. I don't care how many test cases can kill (as long as it's killed) but would like to focus more on survive mutants.

[Freeze777]

Any updates on this one?