Coverage lost if the only changes in a file are new lines / line breaks / line merges

[pjmartos]

Easily reproducible with this project: https://github.com/pjmartos/skippy-nochanges-test

Steps

1. Clone the repository (requires Maven + Java 17)
2. Run mvn clean install, make sure the one unit test in the project gets executed
3. Once the build finishes, check the coverage report target/site/jacoco/index.html; the only class in the project has a 100% coverage
4. Add a new line inside src/main/java/com/pjmartos/Main.java, anywhere except around the last closing curly brace. Alternatively, split the contents of one line into two or more lines, or merge two or more lines into a single line
5. Run mvn clean install once again, make sure the one unit test in the project is now skipped
6. Once the build finishes, check the coverage report target/site/jacoco/index.html; the only class in the project now has 0% coverage

Swapping Imports, adding whitespace or changing indentation doesn't seem to be an issue. Also, if all you do is adding one or more new lines right before and/or right after the last closing curly brace, the coverage stays correct. I suspect this is because none of those changes would alter line numbers of executable code. I guess line numbers are important from bytecode's perspective.

Skippy will skip a test (and consequently won't cause the coverage data of any exercised production classes to be updated) if the bytecode of the current file and the bytecode used for computing coverage the last time for that same file are equivalent, stripping debug information. This works fine, no issues with that.

Unfortunately, JaCoCo won't be able to find any coverage for the new "real" bytecode of the class.

• https://github.com/jacoco/jacoco/blob/v0.8.12/org.jacoco.core/src/org/jacoco/core/analysis/Analyzer.java#L107
• https://github.com/jacoco/jacoco/blob/v0.8.12/org.jacoco.core/src/org/jacoco/core/analysis/Analyzer.java#L81-L88

Expected result The test is skipped without impacting code coverage

Actual result The test is skipped but the coverage for the affected file is lost

[fmck3516]

Thanks for the report. I'm going to look into it.

[fmck3516]

I have to take the full byte code as-is into account when computing hashes: https://github.com/skippy-io/skippy/pull/206

Your initial analysis was spot on!

This will resolve

[WARNING] Classes in bundle 'skippy-nochanges-test' do not match with execution data. For report generation the same class files must be used as at runtime.
[WARNING] Execution data for class com/pjmartos/Main does not match.

Another issue I still haven't figured out: I have to run mvn clean install twice for accurate coverage data to show up in the report. That might be a JaCoCo configuration issue. Still working on that part.

[fmck3516]

This project is no longer being actively maintained due to lack of available time.