Coverage drop when updating from 0.7.5 to 0.7.6

[mattsilber]

Describe the bug Coverage in one of our Android projects drops significantly when upgrading the org.jetbrains.kotlinx:kover-gradle-plugin from 0.7.5 to 0.7.6

Errors None

Expected behavior No significant change in coverage between patch version updates

Reproducer ~Aiming to reproduce this on a public repo; will update this post with the results~ Dependabot already picked it up and the coverage drop was only 4.18%. Still pretty significant, but nothing compared to the percentages we're seeing in the project that led to this report

Reports The following is the output of running 0.7.4 through 0.7.6 on the same commit; the only change being the Kover plugin version:

Kover Version 0.7.4: Class: 73.7% (14/19) Method: 74% (71/96) Branch: 36.8% (111/302) Line: 79.2% (490/619) Instruction: 78.6% (2367/3012)

Kover Version 0.7.5: Class: 73.7% (14/19) Method: 74% (71/96) Branch: 35.8% (108/302) Line: 79.2% (490/619) Instruction: 78.6% (2367/3012)

Kover Version 0.7.6: Class: 73.7% (14/19) Method: 41.7% (40/96) Branch: 4.3% (13/302) Line: 31.5% (195/619) Instruction: 30.6% (921/3009)

Environment

• Kover Gradle Plugin version: 0.7.6
• Gradle version: 8.4 and 8.5
• Kotlin project type: Kotlin/Android
• Coverage Toolset (if customized in build script): Not specified by project
• Other context important for this bug:

[shanshin]

Hi, please try to collect coverage locally for the reproducer.

By these commands, I collected the coverage locally:scratchoff:koverHtmlReportDebug, :scratchoff:koverXmlReportDebug and got such coverage In XML (at the end of file)

<counter type="INSTRUCTION" missed="395" covered="2168"/>
<counter type="BRANCH" missed="62" covered="123"/>
<counter type="LINE" missed="109" covered="513"/>
<counter type="METHOD" missed="37" covered="135"/>
<counter type="CLASS" missed="2" covered="21"/>

And in HTML

If you get the same, then there may be some problem when uploading coverage to codecov or in support of codecov

[mattsilber]

Thanks for getting back to me so quickly on this, just wanted to start by saying I'm a huge fan of kover and the work you all do 🙏

As for the issue, running koverHtmlReportDebug with versions 0.7.5 and 0.7.6 on that same commit does produce the drop, so I can confirm it's happening before the upload of the report to codecov. Sorry I didn't include the following info in the initial report:

Kover Version 0.7.5: Class: 91.3% (21/23) Method: 83.7% (144/172) Branch: 69.7% (129/185) Line: 86.4% (545/631) Instruction: 89% (2291/2575)

Kover Version 0.7.6: Class: 91.3% (21/23) Method: 78.5% (135/172) Branch: 66.5% (123/185) Line: 82.5% (513/622) Instruction: 84.6% (2168/2563)

So the 4.18% drop codecov reports is directly coming from the artifact we upload

Edit to add:

Worth noting these tests are instrumented with Robolectric and I see there was a recent change related to it -- perhaps the addition of jdk.internal.* as an exclusion or the inclnolocationclasses flag had unforeseen downstream effects?

[shanshin]

perhaps the addition of jdk.internal.* as an exclusion or the inclnolocationclasses flag had unforeseen downstream effects?

inclnolocationclasses only works if you explicitly set the use of JaCoCo to measure coverage and generate a report. jdk.internal.* it is also unlikely because it does not overlap with packages in missed classes (checked, it's not the reason).

@zuevmaxim, it seems that in recent versions of the agent, a check has been added, which in some cases has a false positive. Before running the tests locally, you should delete the line buildTypes.debug.testCoverageEnabled true from the scratchoff/build.gradle file so that the JaCoCo agent is not used.

[shanshin]

An example of a falsely missed class: com.jackpocket.scratchoff.views.ScratchableRelativeLayout this constructor is called and in the previous version it was marked as covered.

[shanshin]

Fixed in 0.8.0