JavadocCollector facing issues when runs under Java11

[Den-Rimus]

Question Posting this just in case you might have ideas about the core of the problem or on how to fix this.

While preparing our Android library project for migrating to AGP7.0 (which requires JDK11), we faced the following issue: generating Javadoc documentation by dokka with Gradle wrapper supplied with JDK11 barely gives any output - just some *.css and *.js files and couple of *.png resources (please see screenshots).

• How we run dokka: from command line using gradle wrapper: ./gradlew dokkaJavadocCollector in project root.
• How we switch between different JDK versions: export JAVA_HOME=$(/usr/libexec/java_home -v 1.8) or export JAVA_HOME=$(/usr/libexec/java_home -v11). ./gradlew -version then gives us

------------------------------------------------------------
Gradle 7.2
------------------------------------------------------------

Build time:   2021-08-17 09:59:03 UTC
Revision:     a773786b58bb28710e3dc96c4d1a7063628952ad

Kotlin:       1.5.21
Groovy:       3.0.8
Ant:          Apache Ant(TM) version 1.10.9 compiled on September 27 2020
JVM:          1.8.0_292 (AdoptOpenJDK 25.292-b10)
OS:           Mac OS X 11.5 x86_64

or

------------------------------------------------------------
Gradle 7.2
------------------------------------------------------------

Build time:   2021-08-17 09:59:03 UTC
Revision:     a773786b58bb28710e3dc96c4d1a7063628952ad

Kotlin:       1.5.21
Groovy:       3.0.8
Ant:          Apache Ant(TM) version 1.10.9 compiled on September 27 2020
JVM:          11.0.11 (AdoptOpenJDK 11.0.11+9)
OS:           Mac OS X 10.16 x86_64

respectively.

• How is dokka configured:

    tasks.named("dokkaJavadoc", DokkaTask::class.java) {
        outputDirectory.set("./build/myAwesomeDokkaOutputDir")

        suppressObviousFunctions.set(true)
        suppressInheritedMembers.set(true)

        dokkaSourceSets.configureEach {

            includeNonPublic.set(false)
            skipDeprecated.set(false)
            reportUndocumented.set(true)
            skipEmptyPackages.set(true)
            jdkVersion.set(8)
            noStdlibLink.set(false)
            noJdkLink.set(false)
            noAndroidSdkLink.set(false)
            suppressGeneratedFiles.set(true)

            listOf( // Suppress dokka in matching packages
                """.*\.di\..*""",
            ).forEach { packageRegex ->
                perPackageOption {
                    matchingRegex.set(packageRegex)
                    suppress.set(true)
                }
            }

            listOf( // looks like a bug, maybe refine this part later, when fixed
                    // https://github.com/Kotlin/dokka/issues/1994
                """android\..*""",
                """androidx\..*""",
            ).forEach { packageRegex ->
                perPackageOption {
                    matchingRegex.set(packageRegex)
                    skipDeprecated.set(true)
                }
            }
        }
    }

Now to the sad part: our project is quite big, with about 70 modules, with peculiar Gradle SDK scripts tweaks for different purposes. And I cannot disclose it to public. I tried to re-create issue in new playground-project (just threw in two library-submodules with some classes to document and added dokka setup) but - of course - the issue does not reproduce there.

So far my only hope is that looking at my current dokka output (the non-full, corrupted one) you might have a clue on why it finishes generating documentation with only those files. Maybe, you had this before.

Screenshots This output I get: This is how dokka output looks when I run it under Java1.8:

Installation

• Operating system: macOS 11.5.2
• Build tool: Gradle v7.2/7.1.1/6.8.2
• Dokka version: 1.5.30

Additional context Add any other context about the problem here

[MarcinAman]

Hi, i've got some questions:

• could you try "stock" task, without configuration and see how it goes
• does the same problem occur when running html format in collector (./gradlew dokkaHtmlCollector) and in multimodule (./gradlew dokkaHtmlMultiModule, when running this task please change your configuration to configure DokkaTaskPartial named "dokkaHtmlPartial").
• does this problem persist when you manually configure sourceRoot-s?

[Den-Rimus]

"stock" task, without configuration

Same.

running html format in collector (./gradlew dokkaHtmlCollector)

Kinda same - please see screenshots.

multimodule (./gradlew dokkaHtmlMultiModule)

This one was able to assemble at least something:

navigation.html in root doesn't work though - all links are dead.

does this problem persist when you manually configure sourceRoot-s?

Frankly, could not achieve this. I don't quite understand how to use this setting and what possibilities it brings.

[MarcinAman]

SourceRoots are a way of telling dokka where is the source code located in the module. So for example if you have a kotlin based project the sourceRoot setting should look something like this: sourceRoots.setFrom(file("src/main/kotlin")) and this has to be configured for all subprojects independently (therefore it is a configuration for DokkaTaskPartial in multimodule or DokkaTask in Collector / single module). Could you try adding this line / configuring it accordingly to your project's structure?

[Den-Rimus]

Having added line sourceRoots.setFrom(file("src/main/java")) to every submodule's DokkaTask configuration I get this:

> Task :dokkaJavadocCollector FAILED


FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':dokkaJavadocCollector'.
> lateinit property module has not been initialized

* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':dokkaJavadocCollector'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.lambda$executeIfValid$1(ExecuteActionsTaskExecuter.java:188)
        at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:263)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:186)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:174)
        at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:109)
        at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:51)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:200)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:195)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:75)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:68)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:153)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:68)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:62)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.lambda$call$2(DefaultBuildOperationExecutor.java:79)
        at org.gradle.internal.operations.UnmanagedBuildOperationWrapper.callWithUnmanagedSupport(UnmanagedBuildOperationWrapper.java:54)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:79)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
        at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:74)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:408)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:395)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:388)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:374)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$run$0(DefaultPlanExecutor.java:127)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:191)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:182)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:124)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
Caused by: kotlin.UninitializedPropertyAccessException: lateinit property module has not been initialized
        at org.jetbrains.kotlin.cli.jvm.compiler.CliTraceHolder.getModule(CliTrace.kt:25)
        at org.jetbrains.kotlin.cli.jvm.compiler.CliKotlinAsJavaSupport.findClassOrObjectDeclarations(CliKotlinAsJavaSupport.kt:119)
        at org.jetbrains.kotlin.asJava.finder.JavaElementFinder.findClassesAndObjects(JavaElementFinder.kt:71)
        at org.jetbrains.kotlin.asJava.finder.JavaElementFinder.findClasses(JavaElementFinder.kt:57)
        at org.jetbrains.kotlin.asJava.finder.JavaElementFinder.findClass(JavaElementFinder.kt:46)
        at com.intellij.psi.impl.JavaPsiFacadeImpl.doFindClass(JavaPsiFacadeImpl.java:96)
        at com.intellij.psi.impl.JavaPsiFacadeImpl.findClass(JavaPsiFacadeImpl.java:73)
        at com.intellij.psi.impl.PsiElementFactoryImpl.createTypeByFQClassName(PsiElementFactoryImpl.java:352)
        at com.intellij.psi.PsiType.getTypeByName(PsiType.java:161)
        at com.intellij.psi.PsiType.getJavaLangObject(PsiType.java:173)
        at com.intellij.psi.impl.PsiClassImplUtil.getSuperTypes(PsiClassImplUtil.java:841)
        at com.intellij.psi.impl.source.PsiClassImpl.getSuperTypes(PsiClassImpl.java:215)
        at org.jetbrains.dokka.base.translators.psi.DefaultPsiToDocumentableTranslator$DokkaPsiParser$parseClasslike$2.invokeSuspend(DefaultPsiToDocumentableTranslator.kt:212)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.internal.ScopeCoroutine.afterResume(Scopes.kt:33)
        at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:102)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

[MarcinAman]

Tbh i have no idea why this is happening but i assume that dokka doesn't get correct souresets and/or sourceroots from kotlin plugin. As this is an internal project imho the only way to debug it is to publish yourself a custom dokka build and use IJ's debugger to see what are the values. The easiest will be to add a breakpoint in SingleModuleGeneration

[Den-Rimus]

@MarcinAman greetings again! We're still facing the issue - now with new 1.7.0 dokka. This time we have the project to reproduce at least something: as you mentioned, we tried specifying sourceRoots and this gives an error:

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':dokkaJavadocCollector'.
> lateinit property module has not been initialized

* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':dokkaJavadocCollector'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.lambda$executeIfValid$1(ExecuteActionsTaskExecuter.java:188)
        at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:282)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:186)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:174)
        at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:109)
        at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:51)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:200)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:195)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:75)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:68)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:153)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:68)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:62)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.lambda$call$2(DefaultBuildOperationExecutor.java:79)
        at org.gradle.internal.operations.UnmanagedBuildOperationWrapper.callWithUnmanagedSupport(UnmanagedBuildOperationWrapper.java:54)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:79)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
        at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:74)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:402)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:389)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:382)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:368)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$run$0(DefaultPlanExecutor.java:127)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:191)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:182)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:124)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:61)
Caused by: kotlin.UninitializedPropertyAccessException: lateinit property module has not been initialized
        at org.jetbrains.kotlin.cli.jvm.compiler.CliTraceHolder.getModule(CliTrace.kt:25)
        at org.jetbrains.kotlin.cli.jvm.compiler.CliKotlinAsJavaSupport.findClassOrObjectDeclarations(CliKotlinAsJavaSupport.kt:119)
        at org.jetbrains.kotlin.asJava.finder.JavaElementFinder.findClassesAndObjects(JavaElementFinder.kt:62)
        at org.jetbrains.kotlin.asJava.finder.JavaElementFinder.findClasses(JavaElementFinder.kt:47)
        at org.jetbrains.kotlin.asJava.finder.JavaElementFinder.findClass(JavaElementFinder.kt:36)
        at com.intellij.psi.impl.JavaPsiFacadeImpl.doFindClass(JavaPsiFacadeImpl.java:91)
        at com.intellij.psi.impl.JavaPsiFacadeImpl.findClass(JavaPsiFacadeImpl.java:69)
        at com.intellij.psi.impl.PsiElementFactoryImpl.createTypeByFQClassName(PsiElementFactoryImpl.java:360)
        at com.intellij.psi.PsiType.getTypeByName(PsiType.java:159)
        at com.intellij.psi.PsiType.getJavaLangObject(PsiType.java:171)
        at com.intellij.psi.impl.PsiClassImplUtil.getSuperTypes(PsiClassImplUtil.java:798)
        at com.intellij.psi.impl.source.PsiClassImpl.getSuperTypes(PsiClassImpl.java:220)
        at org.jetbrains.dokka.base.translators.psi.DefaultPsiToDocumentableTranslator$DokkaPsiParser$parseClasslike$2$invokeSuspend$$inlined$with$lambda$3.invoke(DefaultPsiToDocumentableTranslator.kt:206)
        at org.jetbrains.dokka.base.translators.psi.DefaultPsiToDocumentableTranslator$DokkaPsiParser$parseClasslike$2.invokeSuspend(DefaultPsiToDocumentableTranslator.kt:225)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.internal.ScopeCoroutine.afterResume(Scopes.kt:33)
        at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:102)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
        Suppressed: kotlin.UninitializedPropertyAccessException: lateinit property module has not been initialized
                ... 23 more

project to clone to reproduce can be found here

1. use java 11: export JAVA_HOME=$(/usr/libexec/java_home -v11)

denysdrobiazko@Denyss-MBP ~ % java -version
openjdk version "11.0.11" 2021-04-20
OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode)

2. run ./gradlew/dokkaJavadocCollector

Hope you can give some assistance 🙏 , because providing relevant API reference is a substantial part of release process of our product and we kind of bounded with dokka here - since we use Kotlin.

[MarcinAman]

Hi, I am no longer a involved in dokka project but @IgnatBeresnev can help 👍

[IgnatBeresnev]

Hi @Den-Rimus! How did you fix it last time? :) The info might help

I'll have a look in the upcoming days, and thank you for the reproducer, it makes things much easier!

[Den-Rimus]

@IgnatBeresnev the thing is - we didn't fix it last time )

At that time we were trying to update to new AGP and to switch to Java11. Dokka blocked us from doing so - so we postponed.

Now we cannot postpone anymore, because we need newer Kotlin to bump one of our dependencies, and we need to bump up the Java version for that.