Gradle Dokka build intermittently hangs

[ianbrandt]

Describe the bug Occasionally my gradlew dokkaHtmlMultiModule CI job hangs indefinitely.

Usually the task completes in just over two hours. When it hangs, I've let it run for 12+ hours before finally killing it.

Expected behaviour The task always completes.

Screenshots I'll attach consecutive thread dumps of the hung Gradle Daemon process.

To Reproduce The issue is intermittent, and I have not been able to reproduce it locally. It would seem the thread dumps will need to be the starting point for troubleshooting.

Dokka configuration

 withType<DokkaMultiModuleTask>().configureEach {
     notCompatibleWithConfigurationCache(
         "Inputs not currently configuration cache compatible."
     )

     moduleName.set("My Project")
 }

 dokkaHtmlMultiModule.configure {
     pluginConfiguration<DokkaBase, DokkaBaseConfiguration> {
         customAssets = listOf(
             // Override the logo image by providing a custom version with
             // the same name as the default.
             rootProject.layout.projectDirectory.file(
                 "shared/dokka/logo-icon.svg"
             ).asFile,
         )

         footerMessage = COPYRIGHT_AS_HTML_DIV
     }
 }

My 'gradle.properties':

org.gradle.caching=true
org.gradle.configuration-cache=true
org.gradle.configureondemand=true
# 6 hours instead of the default 3 (10800000ms)
org.gradle.daemon.idletimeout=21600000
org.gradle.java.compile-classpath-packaging=true
# 10g+ heap size is needed to prevent OutOfMemoryError during Dokka generation
org.gradle.jvmargs=-Xmx11g -Dfile.encoding=UTF-8
org.gradle.parallel=true
kotlin.daemon.jvmargs=-Xmx4g

Installation

• Operating system: Windows 10
• Build JVM: OpenJDK 11.0.22
• Build tool: Gradle v8.7
• Dokka version: 1.9.20

Additional context My project has just over 400 Java/Kotlin modules that apply the org.jetbrains.dokka plugin via my project convention precompiled script plugin. My build agents have 24 available cores each, so this may be a different issue than #2926.

[ianbrandt]

Thread dumps:

1. thread_dump.zip
2. thread_redump.zip (~16 min. later)

I diffed the two, and they're nearly identical with the exception of the elapsed times.

Analysis of the first thread dump: https://fastthread.io/my-thread-report.jsp?p=c2hhcmVkLzIwMjQvMDUvOS9lNDc1NjY3MC1hM2E4LTQ4MzktOTRjMi00NzU0NmNiMDJlZjAudHh0LS0xNi0xMC00MQ==&.

[whyoleg]

Hey! Sorry for the delay. I've investigated those dumps some time ago, but forgot to mention the results.

Based on dumps I found, that there is some deadlock happening during rendering of pages somewhere here. I was not able to reproduce it locally with some artificial big project. May be it is OS or specific setup dependent. Probably the issue could be related to https://github.com/Kotlin/kotlinx.coroutines/issues/3982 or https://github.com/Kotlin/kotlinx.coroutines/issues/3983, as their description for some reason seems very related to how Dokka is implemented under the hood: coroutines and runBlocking is used here and there with no good structure.

So, I do have questions:

• Do you still experiencing this problem? If so, do you have a workaround?
• Is it possible to somehow reproduce the issue locally, or it's CI only?
• Is there an access to the project to check it locally somehow?

[ianbrandt]

Thank you for looking into this!

Our last hang in CI was on July 9, so it seems to be occurring less frequently for whatever reason. Our project and CI environment evolves fairly regularly. I can't think of a specific change that stands out as something that would've mitigated or eliminated the hangs.

We don't generally run our docs task locally because it takes about an hour and a half to execute. I'll start running it at the end of each day to see if I can reproduce a hang locally.

Unfortunately, this isn't a project I can share publicly.

The Kotlinx Coroutines issues are interesting in general, so thank you for the references. I'll definitely be following those.