boxed
Messed up output in CI logs
I have mutmut running inside the CI process: https://travis-ci.org/sobolevn/heisenbug-2019/builds/501413412#L573
But, the output is not fully shown:
2.24s$ poetry run mutmut run || true
- Mutation testing starting -
These are the steps:
1. A full test suite run will be made to make sure we
can run the tests successfully and we know how long
it takes (to detect infinite loops for example)
2. Mutants will be generated and checked
Mutants are written to the cache in the .mutmut-cache
directory. Print found mutants with `mutmut results`.
Legend for output:
🎉 Killed mutants. The goal is for everything to end up in this bucket.
⏰ Timeout. Test suite took 10 times as long as the baseline so were killed.
🤔 Suspicious. Tests took a long time, but not long enough to be fatal.
🙁 Survived. This means your tests needs to be expanded.
mutmut cache is out of date, clearing it...
1. Running tests without mutations
⠸ Running... Done
2. Checking mutants
The command "poetry run mutmut run || true" exited with 0.
I guess, that is caused by animation. My proposed solution is to include --no-interaction flag to run command. In this case it will just out everything on new lines which is quite useful for CIs.
How about instead adding a flag --ci that has a totally different output scheme? Something that is more like a log format:
Running tests without mutation
Tests without mutation done
Checking mutants
Testing mutant 1
Mutant 1 survived
Testing mutant 2
Mutant 2 was killed
...etc
?
The .mutmut-cache file is the only artifact I guess? If you run in CI you should probably keep that and try to reuse it for subsequent runs.
Heh. "Scheduled" is amusing. This project is something I do on my limited free time and it contends with many many of my other projects.
@boxed I've been testing my code with mutmut on github actions and run into the same problem recently. I'd love to give it a try and see if I'll manage to create the "ci output" functionality.
