Kotlin
Gradle Runner 2.0
Background
Dokka's current Gradle plugin has many problems:
- It is not idiomatic, not up to modern standard and does not follow best practices (see #2700).
- It's very difficult to use and confusing to configure, especially for beginners (confirmed by our UX researches and user interviews)
- It has a lot of internal technical debt, as multiple generations of maintainers worked on it without passing down the knowledge, and it wasn't looked after much this past year.
- There are things that do not cause trouble per se, but that we'd want to change given the chance (for example, Gradle integration tests).
Trying to improve the current plugin and resolve problems gradually proved to be difficult as it's gets overwhelming quick. Moreover, with this many changes needed, it would lead to multiple releases with noticeably broken backward compatibility.
At this point it seems like the best course of action is to just rip the bandaid off, and to redesign and reimplement the Gradle plugin from scratch, taking into consideration all of the experience and knowledge we have now.
This issue exists to keep track of progress and to provide updates to the community.
High-level goals
- Redesign the configuration DSL:
- Use the knowledge we got from conducting UX researches and user interviews.
- Comb through existing issues with the configuration tag and take them into account
- Make sure the DSL respects project isolation and is configuration cache compatible from the get-go
- Address our internal wants and wishes
- Make sure the new plugin is idiomatic and follows best practices, both on the surface and under the hood
- Significantly improve or even re-write user-facing documentation.
We won't be able to implement all feature requests and fix all the bugs, but we'll make sure to lay a solid foundation to address them in the future.
High-level plans
We plan to start with designing the new configuration DSL, and it'll take some thinking time and rounds of internal and external review. It's important to get it (mostly) right the first time around, as we want it to be very stable for at least the first couple of major releases after it's out.
As for re-writing the plugin internals, a regular contributor @aSemy started working on it in January (#2839), which grew into a separate project called Dokkatoo (double pun intended). Dokkatoo has been adopted by a number of big projects (including Gradle itself), and has proved itself to be very idiomatic and modern. We hope Dokkatoo will be contributed upstream into Dokka, laying the very foundation for the Gradle plugin 2.0.
We also plan to test the new plugin on a wide range of projects and platforms, with various Kotlin and Gradle versions, and to provide migration guides -- all to ease the transition as much as possible.
More specific details and plans will follow.
Milestone: milestone/28
Hi, good luck with this endeavor!
Apart from what was already mentioned, my biggest pain point with Dokka is the high memory usage. We have had to increase the Gradle heap space on all projects which use Dokka.
We switched all our projects to Dokkatoo, mostly because of build cache and configuration cache support (which decrease memory usage by a lot, since Dokka is invoked a lot more rarely).
If that is of any help, we would be interested in using our projects to test future experimental versions of Dokka.
I want to further process the documentation so dokka should export the docs as json (whatever) file and then run the html file generator on the json file. This would also allow you to intercept the generation and add your own transformer.
One use-case: I want to dynamically generate a class diagram and add it to the documentation, so at the moment I need to call the Gradle task by myself, but it should automatically wired using inputs/outputs.
While not directly related to the Gradle plugin, it could be relevant when considering a new configuration API or intermediate output format. One thing I'd like to see Dokka support in the future is a redesigned platform / source set selector. See my comment here for how I envision this working ideally.
"it just works" configuration
I want to be able to apply Dokka to a project and it just works, with sensible defaults requiring no configuration.
Performance
Currently Dokka is still quite slow/heavy. This makes it difficult to recommend using Dokka to generate a Javadoc JAR, because some projects publish to Maven Local for testing, and Dokka makes this step too slow.
Split out Dokka Configuration
It would be helpful to split out the Dokka Configuration classes into a separate dependency.
Currently Dokkatoo requires DGP as a compileOnly dependency and only uses them in the isolated Gradle Worker. Using an implementation dependency doesn't work, because it intrinsically links Dokkatoo to a specific Dokka version (which at the moment is configurable - Dokkatoo can run almost any Dokka version) and it would pollute the Gradle buildscript classpath with Dokka classes.
Because Dokkatoo uses a compileOnly dependency on Dokka this increases the risk of accidentally using a 'missing' class during the Dokkatoo Gradle plugin configuration (which can be caught by testing, but I'd prefer not to).
Machine-readable output
+1 for dumping machine-readable output https://github.com/Kotlin/dokka/issues/621
This could also integrate better with Knit https://github.com/Kotlin/kotlinx-knit
Split up Dokka Generator steps
It suspect would be helpful to split up the Dokka Generator into more discrete steps. These steps could then be done with separate Gradle tasks, which would aid with caching/performance/parallelization.
For example, split up the sources/classes/dependencies analysis from markup generation, and store the analysed files in a directory, to be loaded later by the markup generator.
Dokka subproject Module vs aggregating project Publication configuration options
Clarify/simplify some of the Dokka format configuration options. For example, which HTML output options are applicable to a subproject (a Dokka Module) vs an aggregated publication? Ideally the buildscript configuration DSL should only show options that will do something (currently the behaviour can be unclear https://github.com/adamko-dev/dokkatoo/issues/111)
More & automatic external source links
Automatic & more comprehensive external source links. Currently Dokka has some automatic detection, e.g. if it's an Android project then enable the Android external source link. But it would be great if Dokka checked all of the dependencies for a project, and had built-in configuration for enabling external source links for all of the Kotlin libs (like Serialization, Coroutines, etc).
I integrated this with a multi-module project w/o problems, but then tried integrating with a much more complex Android multi-module multi-flavor application and it is hitting all sorts of interesting issues with compiling a releaseInt version which is missing a resource (but we never build it). Would be nice to be able to tell it to run against a specific flavor when generating documetation. The current Dokka plugin appears to be trying to build all flavors...
EDIT: been meaning to update this. I fixed the issues with the extra flavor and also tried excluding it from all sourcesets....but Dokka at the root module still tries compiling it. I'll bet that's why Dokka is considered slow...
Dokka 2.1.0 is out, which means the DGPv2 plugin is enabled by default. So, I finally have the honour of closing this issue: Dokka Gradle plugin is fixed!
I'm sure it's just another version release for most people, but for me, it represents a milestone of my life. In case you didn't know, DGPv2 is based on Dokkatoo, which I started as a hobby (or, more accurately, a yak-shaving) project. My efforts were noticed and encouraged, and JetBrains hired me to continue developing DGP. A couple of years later I'm glad, astonished even, to say I've finally found a job I like.
I'm enormously grateful to @IgnatBeresnev for his recognition and encouragement. Huge respect and thanks to @whyoleg for becoming the Dokka lead and keeping DGP development on track. To everyone in the Dokka, Kotlin Build Tools, and Kotlin teams: thanks for the help, support, and code reviews. And especially thanks to all the users who give feedback, make bug reports and feature requests, and make pull requests - the open source community is great!
When I look back on Dokkatoo, two things stand out. The first is the high point, which was when Gradle decided to use Dokkatoo to generate Gradle docs - I had written a plugin good enough for Gradle itself! The second is just how confusing the name was in meetings. I laughed every time someone said "Dokkatoo" and someone else thought they said "Dokka 2" - exactly what I wanted!
So what's next for Dokkatoo? Frankly, it's obsolete - instead, just use DGP. When I have time I'll push out a final release asking everyone to migrate to DGP. However, perhaps it doesn't have to be the end. I would like to use the repo as a hub for contributor-supported Dokka generator plugins, like Markdown and Jekyll. If you're interested, please get in touch (in kotlinlang Slack, or open an issue on Dokkatoo).
For DGPv2, this release is just the beginning. There's still more docs to write, features to add, and bugs to fix. As always, we want to hear your feedback!
- Please report any feedback or problems https://kotl.in/dokka-issues
- Chat with the community visit
#dokkain https://kotlinlang.slack.com/ (To sign up visit https://kotl.in/slack)