Switching to using the JaCoCo JVM Agent

[shanshin]

We have decided to transition to using the JaCoCo agent for Kotlin Coverage.

As part of this change, the current IntelliJ agent, used in the existing Kover Gradle and Maven plugins as well as the CLI, will be deprecated. This will be carried out as part of the transformation of Kover from a separate plugin into a part of Kotlin.

Motivation

Over time, IntelliJ agent differs less and less from JaCoCo and after supporting inline functions, the JaCoCo agent will be equivalent to the IntelliJ agent in its set of capabilities. It doesn't make sense to support two different agents with different infrastructure and incompatible execution file formats. Using another agent forces to use incompatible tools (like reporter and plugins), change build scripts and plugins and pipelines on CI.

Instead of providing alternatives with mandatory migration to products with a different workflow, we decided to integrate as much as possible with an existing and proven product, while adding additional features that will improve the experience with Kotlin.

Changes in approach

• The JaCoCo agent will replace the IntelliJ agent
• XML reports will generating using the JaCoCo distribution
• JaCoCo HTML report will be generating via the JaCoCo distributive
• a Kotlin-specific HTML report generator will be built on top of the JaCoCo reporter
• a Kotlin Coverage report for exporting data will be built on top of the JaCoCo reporter
• these reporters will also consume execution files generated by the JaCoCo agent
• these reporters will be distributed separately in the form of CLI and kotlin JVM dependency
• if necessary, we will create a fork of JaCoCo to implement bug fixes or add features specific to Kotlin. To prevent agents from starting to disperse, we are simultaneously creating a PR in JaCoCo so that these changes become available to the entire community

Action points

• analyze open bugs on Intelij JVM Agent, check them on the current JaCoCo master build. If the problem does not reproduce in JaCoCo, then mark it as a task to close, if reproduced then create the corresponding bug in JaCoCo.
• compare ways to configure agents (for example, filters) so that when switching to JaCoCo, the same configuration gives the same report
• compare the coverage details between agents, if the behaviour of JaCoCo is very different for some language constructs and this is not technically justified, then we should start a design task as a potential change for the next version of JaCoCo.
• deprecate the use of Kover Coverage Tool in Kover Gradle Plugin and change Coverage Tool in Kover Maven Plugin
• design Kover-specific HTML report and Kover Coverage report
• implement reporter for Kover-specific HTML report and Kover Coverage report on top of the JaCoCo reporter
• implement Kover CLI with Kover reporter, JaCoCo reporter and JaCoCo offline instrumenter inside
• add integration with Android Gradle Plugin to get execution report after running Android instrumentation tests to include them in the report; add synchronization of the JaCoCo version between AGP

Migration to Kotlin Gradle Plugin

The transition to JaCoCo will be carried out in parallel with the migration of logic to KGP.

It should be completed before the migration of Kover Gradle Plugin to Kotlin Gradle Plugin, however, some work may continue after the migration (for example, creating Kotlin-specific reports)

[turkcankeskin]

Hi,

I understand from here that xml reports will be in jacoco format and I can easily provide this xml to sonarqube. I have been trying to integrate kover sonarqube to my kotlin project for 1 week but sonarqube cannot read kover report xml at all.

I don't understand if I couldn't do this, but in the current version of Sonarqube, jacoco is recommended for Kotlin. link

sonar.coverageReportPaths or sonar.coverage.jacoco.xmlReportPaths cannot read kover report xml.

[shanshin]

@turkcankeskin, hi, does switching to JaCoCo in Kover help you? It is possible to use the JaCoCo reporter in Kover plugin right now.

[mannodermaus]

By switching to the JaCoCo agent, won't you make Kover a lot worse at customization? In my experience, most of the reports.filters DSL is useless when running Kover against JaCoCo since it has many limitations that were a draw for us to switch to Kover and the IntelliJ agent in the first place. The famous issue of "exclude code marked with an annotation other than *Generated*" comes to mind. In our case, coverage percentage drops from the high 60s to around 8% just because of all the generated and third-party code that we cannot reliably exclude using JaCoCo.

You mention how "if necessary, we will create a fork of JaCoCo to implement bug fixes or add features". Will this include an effort to achieve feature parity with the customizability provided with the IntelliJ agent?

[shanshin]

The famous issue of "exclude code marked with an annotation other than Generated" comes to mind. In our case, coverage percentage drops from the high 60s to around 8% just because of all the generated and third-party code that we cannot reliably exclude using JaCoCo.

The specific is that the filtering of reports does not occur at the JVM agent level.

There is a separate code that turns the agent's measurements into a human-readable form: a percentage of coverage, an XML/HTML report - it's the reporter.

To provide filtering functionality similar to the current Kover functionality, a reporter will be created that will use the results of the JaCoCo agent measurements.

These are points

 - a Kotlin-specific HTML report generator will be built on top of the JaCoCo agent
 - a Kotlin Coverage report for exporting data will be built on top of the JaCoCo agent

in the above plan.

[mannodermaus]

Understood. Thank you for the detailed response! Looking forward to seeing what's in store for Kover.

[TuanLe-Buuuk]

Hi @shanshin , can I check if this ticket will fix the issue relates to the annotation mentioned in here https://github.com/Kotlin/kotlinx-kover/issues/686