stacks-core
stacks-core copied to clipboard
Mutation testing - logger tracking missing/timeout/unviable tests
Applicable issues
- fixes #3806
- #4177
Additional info (benefits, drawbacks, caveats)
Solutions and Recommended Usage
Cargo-mutant does an incremental build for each mutation, that’s why this solution comes with the implementation of tracking the output and only updating it with the differences, to run 10-20 mutants instead of 4000.
CI YML File: Are linked with the actions in the actions' repo https://github.com/stacks-network/actions/pull/3.
Should be executed with every PR, as it tests only the updated or newly added functions in the PR's commits.
Handling Exit Codes (docs)
In the logger
workflow they are updating the place they belong to.
Checklist
- [x] Fix mutation testing on packages to run on the supposed functions, not just on some
- [x] PR CI checker to have all functions tested
- missed/unviable/timeout outputs result in an error and stop the PR from being merged
- outputs the functions that have to be tested
- [x] On merge on main/develop/next branch override (by appending) current output with the merged one
- [x] Mutation Testing Coverage for the already existing functions
- [ ] Mutation testing triggering on git diff that didn't affect the functions, but affected the tests. (if the function already exists and is not modified and the tests that use it are created/updated, the functions will not be included in the cargo mutants)
- [ ] Mutants initial sync completed for
develop
andnext
branches
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Comparison is base (
1f5cc3f
) 62.42% compared to head (53cc9c4
) 82.61%. Report is 20 commits behind head on develop.
Additional details and impacted files
@@ Coverage Diff @@
## develop #4179 +/- ##
============================================
+ Coverage 62.42% 82.61% +20.18%
============================================
Files 401 401
Lines 286383 288405 +2022
============================================
+ Hits 178775 238256 +59481
+ Misses 107608 50149 -57459
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Feedback from PR meeting - let's move to draft for now.
Related discussion in Nakamoto slack channel.