muter
muter copied to clipboard
muter-mutation-testing
Mutation Switching
Heya folks,
I'm a big fan of the framework, I've run it on a few projects thus far and the feedback it gives is great! My problem is that it's very slow. I just had to run mutation tests on a large project and it was supposed to take about 15 days to finish, I actually let it run and it died after 4 days of running because it failed to compile too many times in a row after mutating.
SO, many other mutation testing frameworks have a concept of "Mutation Switching" which drastically speeds up the process by essentially generating code that uses global state to know which mutant is running, but really just has to compile once, then execute. Here's an article that goes into details: https://stryker-mutator.io/blog/announcing-stryker-4-mutation-switching/
I recognize adding this is likely a large effort, and possibly something that has already been discussed (although a cursory glance of your issues doesn't reveal it).
Long story short this is a combination feature request/sort of a declaration of intent, I'd love to fork the repo and get started on the concept. Is this something the Muter community is interested in? Have any conversations around this been started?
Hi @Tyler-Keith-Thompson, thanks for reaching out!
The feature you are referring is called mutant schemata and is for sure under our radar!
This is super complex and we are trying to figure out which one works best for swift and it's tooling.
You are 100% correct that this is a huge effort, that brings tons of questions, like: how do you inject the next mutation, without the need to recompile the entire project?
If you have any POC or ideas, please send it over, I would love to hear it!
A related idea we've had is to use code coverage to be able to see which lines can be omitted from mutation testing because they're not even covered. Theoretically, if you track coverage for each test case, you'd know that two tests exercise mutually exclusive code, and therefore could have their mutations added in the same compile. I think.
