Append generation and publication of Gradle catalog to the release process

[elect86]

Hello,

as titled, I'd like to append the generation and publication of Gradle catalog to the release process

Would this be ok for you? What's the best way to achieve this?

[ctrueden]

@elect86 Yes, sounds good. I'm in the midst of some server migration work right now, but resolving this Gradle catalog goal is on my shortlist after that. Will try to respond in technical detail before next Friday. The timing is good because we need to make a release of pom-scijava 34.0.0 soon; hopefully we can automate the Gradle catalog updates in time to benefit from that.

[ctrueden]

@elect86 I took a look at the scijava-catalog. But it is not clear to me what needs to be done:

1. How do I test this? This plugin looks unfinished, with no tests or examples illustrating how it should be used (the only two test sources in functionalTest and test respectively are commented out).

2. Which artifact(s), if any, should be published where? Running ./gradlew jar produces ./build/libs/gradle-catalog-33.2.0.jar, with the various catalogs as well as the sciJava.SciJavaCatalogPlugin itself. (Does this package matter? If not, it should probably be org.scijava.gradle or org.scijava.catalog or similar, for consistency with the rest of the SciJava components.) Where should this gradle-catalog-x.y.z.jar be published? To OSS Sonatype along with pom-scijava? If so, what should the published gradle-catalog-x.y.z.pom look like? How does Gradle handle this? Or does it not need to be published at all apart from here on GitHub?

The parsing logic of buildSrc/src/main/kotlin/core/parsing.kt is brittle, parsing line by line rather than using the XML DOM. It would also probably simplify our lives to run mvn help:effective-pom first and then parse the output, since all properties etc. would then be interpolated. What do you think?

If you could give me a primer on what publication of a Gradle catalog entails, that would help me to get started on integrating this work better into pom-scijava and its releases. Thanks!

[ctrueden]

@elect86 Still looking for guidance on how to move this forward. I would love to include it soon, but I don't know how to test it. Or maybe @skalarproduktraum, do you know?

[elect86]

Sorry, I wanted to tackle this but I still haven't got the time, atm I have to work on imgui for scenery

[ctrueden]

@elect86 No worries! I'll wait then till you have time to give me some guidance. No rush. 😄

[ctrueden]

@elect86 Here is a kscript that prints out all the dependencies present in the pom-scijava <dependencyManagement> section:

#!/usr/bin/env kscript

@file:DependsOn("org.scijava:scijava-common:2.94.1")

import java.io.File
import java.util.ArrayList
import org.scijava.util.POM
import org.scijava.util.XML
import org.w3c.dom.Element

println("Parsing POM file...")

val pom: POM = POM(File("eff.xml"))

val deps = pom.elements("//project/dependencyManagement/dependencies/dependency")

deps.forEach { dep ->
    val g: String = XML.cdata(dep, "groupId")
    val a: String = XML.cdata(dep, "artifactId")
    val v: String = XML.cdata(dep, "version")
    val exclusionsElement: List<Element> = XML.elements(dep, "exclusions")
    val exclusions: List<Element> = if (exclusionsElement.isEmpty()) emptyList() else XML.elements(exclusionsElement[0], "exclusion")
    println("$g : $a : $v -- # of exclusions = ${exclusions.size}")
}

You can generate the eff.xml by running:

mvn -B help:effective-pom | sed '/^[^ \t<]/d' | sed '/^$/d' > eff.xml

which interpolates the various POM layers into one flattened POM, with all property values filled in. This is easier to parse, than doing it recursively yourself.

The script leans on the scijava-common library's org.scijava.util.POM (which extends org.scijava.util.XML) utility class for easier handling of XML data via xpath and such. We could do it ourselves via direct calls to the org.w3c.dom library etc., but it would be recapitulating code that SciJava Common already gives us for free, so why not reuse it?

The hope is that the above approach provides a simpler and more robust way of parsing the pom-scijava pom.xml, and code-generating the matching catalog.kt.

[ctrueden]

Here is the generated eff.xml as of 35.1.1+2 (3c3b9ba417f1160fbe9f35f818ce8dcfc6f61512): eff.xml.zip

[ctrueden]

Related idea: according to this blog post, there are benefits to publishing Gradle Module Metadata along with pom-scijava. The gradle-module-metadata-maven-plugin is capable of automating that, so perhaps we should start using it?

See also this Zulip discussion.

[ctrueden]

With cbf7caf5c2b3c96fe266a62aa2e84c8e02f34884, 6388e544626b42f975c77e92c81494be24587146, and e48041ea95f9ebc00de7478ed9c3ca2c3b0b1b0e, support has begun for supporting Gradle platforms and/or catalogs.

Still to do:

1. Remove eff.xml from version control—it needs to be autogenerated from the pom.xml instead.
2. Integrate the Gradle platform/catalog build into the Maven build, so that the platform and/or catalog can be attached as a build artifact, so that it gets deployed to the remote Maven repository along with the POM itself.