Same test suite, but different instruction coverage measurements exist

[hqsz]

Describe the bug Same test suite, but different instruction coverage measurements exist

Expected behavior coverage measurements should be equal

Reproducer

1. clone https://github.com/hqsz/kover-coverage-bug-240910
2. execute koverHtmlReport with oneline : result is in build-with-oneline folder
3. execute koverHtmlReport with multiline : result ins in build-with-multiline folder

Reports With same test suite, but difference instruction coverage

Environment

• Kover Gradle Plugin version: 0.8.3
• Gradle version: 8.6
• Kotlin project type: Kotlin/JVM
• Coverage Toolset (if customized in build script): Kover

[hqsz]

In fact, before reporting this bug, I encountered another similar bug. Due to the company's security issues, I cannot attach entire code. So I tried to reproduce it, but failed.

The first bug discovered was as follows :

---

Describe the bug

• Unexecuted instruction coverage is filled

Expected behavior

• Unexecuted instruction must not be filled

Reports

• Wrong behavior

val nullableVal: Something?

when (someEnum) {
  A -> { // test suite execute all branch and all instruction and it is all green(covered), also if i remove certain branch it changes to red properly
    if (nullableVal != null) {
       doSomething()
    } else {
       doSomething()
    }
  }
  B -> {
     if (nullableVal != null) {
        doSomething() // test suite only covered nullableVal equal null case, but it is marked as green(covered)
     }
  }
}

But if i add else branch it looks normal

when (someEnum) {
  A -> { ... }
  B -> {
     if (nullableVal != null) {
        doSomething() // Despite being the same test suite, this turns to red(not covered)
     } else {
        doSomething() // and this branch is marked as green(looks normal)
     }
  }
}

Environment

• Kover Gradle Plugin version: 0.8.2 (but also 0.8.3 is same)
• Gradle version: 7.6
• Kotlin project type: Kotlin/JVM
• Coverage Toolset (if customized in build script): Kover

[shanshin]

Thanks for the reproducer!

[hqsz]

@shanshin If you need anything, please feel free to tell me 😄

I also have the following questions:

Is it a bug that the null checking inside the if statement branch is marked green even though both the true and false results are not tested? (with upper repository, it is only tested false case) I thought it should be marked in orange (or yellow).

[shanshin]

Is it a bug that the null checking inside the if statement branch is marked green even though both the true and false results are not tested? (with upper repository, it is only tested false case) I thought it should be marked in orange (or yellow).

Yes, it should be yellow. However, it seems that when generating a report, when several bytes of instruction code are on the same line, this is not perceived as branching (there may be confusion with Kotlin hided null-check)

[hqsz]

Thanks for answering!

If there is anything I can contribute, I am happy to do so 😃

[shanshin]

Closed due to the end of IntelliJ Agent development (#720) and the introduction of a new strategy for measuring of Kotlin code coverage (#746).

It is recommended to use JaCoCo, for this purpose it is enough to specify

kover {
    useJacoco("0.8.13")
}

in each project.

Based on our internal testing, this issue does not appear to be reproducible in JaCoCo.