tarpaulin
tarpaulin copied to clipboard
Fail decreasing
This pull request implements the --fail-decreasing
command line option, which allows tarpaulin to return an error if the coverage percentage has decreased, similar to --fail-under
which fails the run if the coverage is under a certain threshold.
Current feature set implemented:
- If --fail-under is used by it self, it should behave like before
- If --fail-decreasing is used by it self, it will return an error if the coverage percentage is decreasing.
- If --fail-under and --fail-decreasing is used together is works as follows, if the coverage is below the threshold, it error out if the coverage is decreasing, if the coverage is above the threshold, the coverage can increase and decrease freely.
I have had a quick look at the tests, but are uncertain how and if I should make a test case for this, becaues it would require multi step tests, a first run, one increasing, one returning the same coverage, and one where it is decreasing.
This fixes: #1411
Note: I'm new to rust, so if you have any changes you want made, there could be some terminoligy, that i'm not fluent in or familier with yet :-)
I have now implemented test cases for the feature, and while doing so found 2 bugs that I fixed.
During test runs I see random test failures in:
- cargo-tarpaulin::integration config_file_coverage
- cargo-tarpaulin::integration follow_exes_down
- cargo-tarpaulin::integration llvm_sanity_test
Should I take a look at them, and submit a seperate PR, i expect it is something similar to what I had to deal with when I was writing the test cases in this PR, there may be more like it ?
Just at work so will look properly later, but I use set_profraw_dir
on the target to stop tests clobbering each other.
I tried that, but couldn't get it to work, ended up using config.set_target_dir() instead, which works nicely.