Switching from Jacoco to Kover is having OOM issues

[DavidCorrado]

Describe the bug So I am trying to convert from Jacoco to Kover but just simply doing that and running this command ./gradlew clean app:koverHtmlReportDebug -PenableKover (Not using this work around to not impact testDebugUnitTests https://github.com/Kotlin/kotlinx-kover/issues/531) The problem is we have over 3000 unit tests and it gives OOM errors. java.lang.OutOfMemoryError

It might be difficult to get an example to replicate this issue. I imagine I am doing something that is causing a large amount of memory to be used via the tests but its really not easy to troubleshoot.

But on top of that if this works on Jacoco would you expect it to work with Kover?

Note if I delete around 20 screens of unit tests the task succeds. I am running this on an Android compose single module app. I think the compose unit tests likely take more memory.

But anyways. I tried to play around with gradle.properties and set parellel to false(org.gradle.parallel,org.gradle.configureondemand,org.gradle.caching) Also I have a 64 GB ram machine so I tried all numbers up to 50 GB of ram and it did not work.

Also I replicated this on CI on gitlab.

1. Any troubleshooting recommendations
2. Can we see about maybe getting the performance to be more comparable to jacoco with memory usage?

Here are some potentially related issues https://github.com/search?q=repo%3AKotlin%2Fkotlinx-kover+memory&type=issues

Thanks for reviewing.

[shanshin]

Hi, could you please clarify where does the OOM appear? In the process in which the tests are running or in the process in which the Gradle daemon is running, and what is the full stack trace of the error?

Also, what do you mean by "switched from JaCoCo to Kover"? Have you used the JaCoCo Gradle plugin before? How was it configured, are the tests running for one module or for several?

[DavidCorrado]

Its erroring when running this task which is within

Output is something like this

CreateAvatarViewModelTest > saveAvatarImage test when avatar and background color are not selected FAILED
    java.lang.OutOfMemoryError at CreateAvatarViewModelTest.kt:175
<===========--> 91% EXECUTING [3m 34s]
> IDLE
> IDLE
> :app:testDebugUnitTest > Executing test co.well.wellapp.ui.views.WellCheckableCardTest
> IDLE
> IDLE
> IDLE
> IDLE
> IDLE
> IDLE
> IDLE
> IDLE
> IDLE
> IDLE
> IDLE
> :app:testDebugUnitTest > 2581 tests completed, 1 failed, 2 skipped

The line of code that OOM is random when the system determines its out of memory

I think per my command having an app: before it that it means its only running on the app module. I really only have that and a very small lint module. I believe I am running the tests only for the app module as my command is app:koverHtmlReportDebug

Yes I was using the jacoco plugin before and am trying to convert to kover

org.jacoco:org.jacoco.core:0.8.12

It is configured to run the same unit tests so testDebugUnitTest

My config looked like this

tasks.withType<Test> {
    configure<JacocoTaskExtension> {
        // Required for JaCoCo + Robolectric
        // https://github.com/robolectric/robolectric/issues/2230
        isIncludeNoLocationClasses = true

        // Required for JDK 11 with the above
        // https://github.com/gradle/gradle/issues/5184#issuecomment-391982009
        excludes = listOf("jdk.internal.*")
    }
}

tasks.register<JacocoReport>("jacocoTestReport") {

    dependsOn("testDebugUnitTest")
    group = "reporting"
    description = "Generate Jacoco coverage reports for the debug build."

    reports {
        xml.required.set(true)
        html.required.set(true)
    }

    val fileFilter = listOf(
        // Auto-Generated by our libraries
        "**/*JsonAdapter.*", // Generated by Moshi
        "**/*_Impl**", // Generated by Room

        // Auto-Generated by Android
        "**/R.class",
        "**/R\$*.class",
        "**/BuildConfig.*",
        "**/Manifest*.*",

        // Manually exclude debug only code
        "**/debugmenu/**", // Don't calculate coverage on the debugmenu
        "**/databaseplugin/**", // Don't calculate coverage on database plugin
        "**/VariantUtil.*" // Don't calculate coverage on VariantUtil
    )

    val kotlinTree = fileTree("${project.layout.buildDirectory.get()}/tmp/kotlin-classes/debug") {
        exclude(fileFilter)
    }
    val javaTree = fileTree("${project.layout.buildDirectory.get()}/intermediates/javac/debug") {
        exclude(fileFilter)
    }

    val mainSrc = "${project.projectDir}/src/main/java"

    val jacocoExecutionData = fileTree(project.layout.buildDirectory.get()) {
        include("outputs/unit_test_code_coverage/debugUnitTest/testDebugUnitTest.exec")
    }

    sourceDirectories.setFrom(files(mainSrc))
    classDirectories.setFrom(files(kotlinTree, javaTree))
    executionData.setFrom(jacocoExecutionData)
}

[shanshin]

And if you try to use JaCoCo in Kover Plugin, the error will not be reproduced?

kover { 
    useJacoco()
}

[DavidCorrado]

FATAL ERROR in native method: processing of -javaagent failed, processJavaStart failed
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
*** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with message Outstanding error when calling method in invokeJavaAgentMainMethod at src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 619
*** java.lang.instrument ASSERTION FAILED ***: "success" with message invokeJavaAgentMainMethod failed at src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 459
*** java.lang.instrument ASSERTION FAILED ***: "result" with message agent load/premain call failed at src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 422
V  [libjvm.dylib+0x4bba98]  jni_FatalError+0x10c
V  [libjvm.dylib+0x61a4c0]  JvmtiExport::post_vm_initialized()+0x268
V  [libjvm.dylib+0x99b97c]  Threads::create_vm(JavaVMInitArgs*, bool*)+0x650
V  [libjvm.dylib+0x4d8abc]  JNI_CreateJavaVM+0x78
C  [libjli.dylib+0xa7e8]  JavaMain+0x100
C  [libjli.dylib+0xd8c4]  ThreadJavaMain+0xc
C  [libsystem_pthread.dylib+0x72e4]  _pthread_start+0x88

Process 'Gradle Test Executor 124' finished with non-zero exit value 134
org.gradle.process.internal.ExecException: Process 'Gradle Test Executor 124' finished with non-zero exit value 134
        at org.gradle.process.internal.DefaultExecHandle$ExecResultImpl.assertNormalExitValue(DefaultExecHandle.java:442)
        at org.gradle.process.internal.worker.DefaultWorkerProcess.onProcessStop(DefaultWorkerProcess.java:146)
        at org.gradle.process.internal.worker.DefaultWorkerProcess.access$000(DefaultWorkerProcess.java:43)
        at org.gradle.process.internal.worker.DefaultWorkerProcess$1.executionFinished(DefaultWorkerProcess.java:99)
        at jdk.internal.reflect.GeneratedMethodAccessor317.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
        at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:44)
        at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:268)
        at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:170)
        at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:148)
        at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:37)
        at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:92)
        at jdk.proxy1/jdk.proxy1.$Proxy161.executionFinished(Unknown Source)
        at org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:221)
        at org.gradle.process.internal.DefaultExecHandle.finished(DefaultExecHandle.java:381)
        at org.gradle.process.internal.ExecHandleRunner.completed(ExecHandleRunner.java:134)
        at org.gradle.process.internal.ExecHandleRunner.lambda$run$2(ExecHandleRunner.java:97)
        at org.gradle.internal.operations.CurrentBuildOperationRef.with(CurrentBuildOperationRef.java:85)
        at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:95)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
        at org.gradle.internal.concurrent.AbstractManagedExecutor$1.run(AbstractManagedExecutor.java:48)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:491)
        at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:503)
Caused by: java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at org.jacoco.agent.rt.internal_aeaf9ab.core.runtime.InjectedClassRuntime$Lookup.defineClass(InjectedClassRuntime.java:134)
        at org.jacoco.agent.rt.internal_aeaf9ab.core.runtime.InjectedClassRuntime.startup(InjectedClassRuntime.java:54)
        at org.jacoco.agent.rt.internal_aeaf9ab.PreMain.premain(PreMain.java:50)
        ... 6 more
Caused by: java.lang.LinkageError: loader 'bootstrap' attempted duplicate class definition for java.lang.$JaCoCo. (java.lang.$JaCoCo is in module java.base of loader 'bootstrap')
        at java.base/java.lang.ClassLoader.defineClass0(Native Method)
        at java.base/java.lang.System$2.defineClass(System.java:2311)
        at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2439)
        at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2416)
        at java.base/java.lang.invoke.MethodHandles$Lookup.defineClass(MethodHandles.java:1843)
        ... 13 more
*** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with message Outstanding error when calling method in invokeJavaAgentMainMethod at src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 619
*** java.lang.instrument ASSERTION FAILED ***: "success" with message invokeJavaAgentMainMethod failed at src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 459
*** java.lang.instrument ASSERTION FAILED ***: "result" with message agent load/premain call failed at src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 422

It repeatedly gives this error rather than running the unit tests

[shanshin]

Is the JaCoCo plugin disabled at the same time?

[DavidCorrado]

I removed all of the existing jacoco code. Do I need to keep that in there?

[shanshin]

I removed all of the existing jacoco code. Do I need to keep that in there?

Is the JaCoCo plugin applied, if it is applied, it is in the list of plugins, for example

plugins {
    kotlin("jvm")
    // ...
    jacoco
}

[DavidCorrado]

plugins { alias(libs.plugins.android.application) alias(libs.plugins.compose.compiler) kotlin("android") alias(libs.plugins.google.services) alias(libs.plugins.firebase.perf) alias(libs.plugins.firebase.crashlytics) alias(libs.plugins.ktlint) alias(libs.plugins.roborazzi) alias(libs.plugins.devtools.ksp) alias(libs.plugins.kotlin.serialization) alias(libs.plugins.kotlin.kover) jacoco } Just added it with the same error. Is there an sample of jacoco working that I can compare to mine?

[shanshin]

Just added it with the same error.

You don't need to add it, on the contrary, I clarified that two plugins should not be used at the same time.

Just judging by the error, it looked like the instrumentation had been done twice. Perhaps JaCoCo is enabled in Android Gradle plugin

[DavidCorrado]

Ahh yeah it was not there. I have these lines of code in my build.gradle that basically seems to be implicitly adding code coverage stuff that conflicts with your library as you say to not do that.

enableUnitTestCoverage = true enableAndroidTestCoverage = true

This was the main problem of useJacoco() not working.

https://developer.android.com/reference/tools/gradle-api/8.3/null/com/android/build/api/dsl/BuildType#setEnableUnitTestCoverage(kotlin.Boolean)

Note without useJacoco it still does not work. Returns OOM issues. I deleted all of my gradle caches(locally to my app and to my user profile on my mac ~/.gradle. Note that with useJacoco it does not give OOM errors

./gradlew clean app:koverHtmlReportDebug -PenableKover
Downloading https://services.gradle.org/distributions/gradle-8.9-bin.zip
.................................................................................................................................
Unzipping /Users/david.corrado/.gradle/wrapper/dists/gradle-8.9-bin/90cnw93cvbtalezasaz0blq0a/gradle-8.9-bin.zip to /Users/david.corrado/.gradle/wrapper/dists/gradle-8.9-bin/90cnw93cvbtalezasaz0blq0a
Set executable permissions for: /Users/david.corrado/.gradle/wrapper/dists/gradle-8.9-bin/90cnw93cvbtalezasaz0blq0a/gradle-8.9/bin/gradle

Welcome to Gradle 8.9!

Here are the highlights of this release:
 - Enhanced Error and Warning Messages
 - IDE Integration Improvements
 - Daemon JVM Information

For more details see https://docs.gradle.org/8.9/release-notes.html

Starting a Gradle Daemon, 1 stopped Daemon could not be reused, use --status for details
Configuration on demand is an incubating feature.

> Task :app:testDebugUnitTest

WellDatePickerDialogTest > DatePickerDialog when screen height is 570 renders view correctly FAILED
    java.lang.OutOfMemoryError
    java.lang.OutOfMemoryError


Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "SDK 34 Main Thread"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Test worker"

> Task :app:testDebugUnitTest FAILED

2595 tests completed, 1 failed, 3 skipped

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:testDebugUnitTest'.
> Process 'Gradle Test Executor 1' finished with non-zero exit value 1
  This problem might be caused by incorrect test process configuration.
  For more on test execution, please refer to https://docs.gradle.org/8.9/userguide/java_testing.html#sec:test_execution in the Gradle documentation.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

BUILD FAILED in 8m 48s
37 actionable tasks: 36 executed, 1 up-to-date

[shanshin]

@DavidCorrado, a new JaCoCo version was recently released, it contains support for inline functions.

if you specify useJacoco("0.8.13"), does it cover the needs of your project?

[shanshin]

Closed due to the end of IntelliJ Agent development (https://github.com/Kotlin/kotlinx-kover/issues/720) and the introduction of a new strategy for measuring of Kotlin code coverage (https://github.com/Kotlin/kotlinx-kover/issues/746).