Add support for Gradle Version Catalogs (Gradle 7.0)

[ReginFell]

Gradle 7.0 was released recently and it introduces a new Central Declaration System as well as a new supported format .toml. https://github.com/gradle/gradle/issues/15352

Would be nice if Dependabot supported this.

[JavierSegoviaCordoba]

I would like that dependabot search for ALL toml files in the project (regardless of the directory or subdirectory the file is in).

[madhead]

As an expansion for this…

I was using a trick with a Versions.kt file in buildSrc hosting all the versions for my project and using it in the build. Take a look at the example in madhead/ImgMacroBot:

buildSrc/src/main/kotlin/Versions.kt

object Versions {
    const val JVM = "11"

    object Plugins {
        const val KOTLIN = "1.4.21"
        const val DETEKT = "1.15.0"
        const val LIQUIBASE = "2.0.4"
    }

    object Dependencies {
        const val KTOR = "1.5.1"
        const val KOIN = "2.2.2"
        const val KOTLINX_SERIALIZATION = "1.0.1"
        const val TGBOTAPI = "0.30.0"
        const val SKIJA = "0.89.18"
        const val MICROMETER = "1.6.4"
        const val LOG4J = "2.14.0"
        const val JACKSON = "2.12.1"
        const val JUNIT = "5.7.1"
        const val MOCKK = "1.10.6"
        const val LIQUIBASE = "4.3.1"
        const val SNAKEYAML = "1.28"
        const val OJDBC11 = "21.1.0.0"
    }
}

And then in any build.gradle.kts:

dependencies {
    implementation(platform("org.apache.logging.log4j:log4j-bom:${Versions.Dependencies.LOG4J}"))
    implementation(platform("io.micrometer:micrometer-bom:${Versions.Dependencies.MICROMETER}"))
 
    api("io.micrometer:micrometer-core")
    api("dev.inmo:tgbotapi:${Versions.Dependencies.TGBOTAPI}")
    api("org.jetbrains.skija:skija-linux:${Versions.Dependencies.SKIJA}")
    …
}

Dependabot seems not to handle that correctly, check the repo's insights: the dependencies list is empty. I don't know if Dependabot actually executes Gradle to parse the dependencies, but seems like it's abilities in parsing them are limited.

Probably, if it executed full-featured Gradle it could handle any scenarios like mine or like the new central declaration system.

[huehnerlady]

@madhead that sounds like a completely different issue. Here we ask for the support of the new gradle feature with toml files and not for the buildSrc workaround. Maybe it makes sense to open a separate issue for that?

[JavierSegoviaCordoba]

Another thing, a Version Catalog can be published, and you can add them to whatever project you want via settings.gradle files:

dependencyResolutionManagement {
    repositories {
        mavenCentral()
        google()
        gradlePluginPortal()
    }

    versionCatalogs {
        create("libs") { from("com.example.catalogs:some-catalog:1.0.0") }
    }
}

So should be great dependabot can search for updates for them.

[madhead]

Take it easier!

My point is not about that particular buildSrc example, it's just an example. I'm trying to add a comment that if only Dependabot actually evaluate the build script and extract the versions from the model, it could support all the scenarios, including this new TOML feature and aforementioned versionCatalogs.

P.S. I don't know how does Dependabot work currently, but it seems like it uses a bunch of regexps to parse the scripts. I understand it could be the only option - evaluating a build scripts needs a Java, which might be unacceptable. It's also impossible to make a PR based on the evaluated model, but it's just an idea.

[tbroyer]

@madhead There's #1164 already. It wouldn't tell Dependabot which file to update though, for automatic pull requests.

[madhead]

@tbroyer, yep, looks exactly what I was talking about! And, yeah, updates are almost impossible with this approach.

[JavierSegoviaCordoba]

Anyway, having dependencies in buildSrc is not only a bad practice, there are unlimited ways to organize them there, so it is really difficult to cover all use cases.

Knowing version catalogs will be the standard way for dependency management, I think there is no sense in searching a way to get it working in buildSrc.

[DanySK]

There's a bit of confusion. There are two different issues at hand:

1. supporting the central declaration system;
2. supporting Gradle fully.

Although 2. (see #1164 as @tbroyer suggests) would likely cover both, adding 1. first would be useful in many cases and easier.

[ReginFell]

Yeah, parsing toml should be relatively simple task since dependabot already has this for Python

[huehnerlady]

I would say the ultimate goal is to support gradle fully. if this could be done in the near future, then no need for the central dependencies. BUT as it does not seem to be near I would say it might be a very good way to migrate towards more support of gradle-kotlin-dsl :)

[huehnerlady]

Is there any update on this? Would be great to have this feature and I thought someone mentioned it should not be tooo much work?

[jurre]

It's not currently on our roadmap, I'm not sure who said that, but it seems like a significant amount of work from a quick glance

[DanySK]

@jurre, the comparison was being made with the proposal for going through the Gradle Tooling API #1164; my feeling is that the amount of work required to support the central declaration in a TOML file would be way smaller...

[jurre]

@jurre, the comparison was being made with the proposal for going through the Gradle Tooling API #1164; my feeling is that the amount of work required to support the central declaration in a TOML file would be way smaller...

Yes, I agree that it seems smaller, but still a significant amount of work that we currently don't have planned, so I can't give a timeline

[DanySK]

I see; thanks for the clarification.

[huehnerlady]

@jurre thanks for the clarification. is there anything we can do to get it on the roadmap asap? 😉

[be1ski]

@jurre we are very sad to hear that 🥲 maybe you have at least an approximate date for the appearance of this feature?

[StefanOltmann]

Sad. Dependabot does not work with Gradle in most use cases.

[jurre]

@jurre we are very sad to hear that 🥲 maybe you have at least an approximate date for the appearance of this feature?

@asciimike would be the best person to help answer that, but I don't think there is much we can commit to at this point

[DanySK]

@StefanOltmann @be1ski one way around the current level of support is to manage dependencies via refreshVersions and automatic upgrades via UpGradle. I'm currently achieving dependabot-like functionality this way, while I wait for support to Gradle to be complete.

[JavierSegoviaCordoba]

I am being forced to generate the TOML files from code inside build.gradle.kts files to get dependabot working.

[Nambers]

Is there any update on this? Would be great to have this kind of feature like supporting the toml or buildSrc in gradle. But my dependabot look like dont get my toml file.

[asciimike]

No updates on timeline, as we don't have the gradle expertise or time at present to properly prioritize this (or #1164, which would be even better). If folks are interested in working on this, let us know.

[dmedinag]

I have to add myself to the interested parties in seeing this happen. TOML catalogs introduce a massive leap forward in dependency management for frameworks developed with gradle and since we're using my team and I have had to say goodbye to Dependabot :( We'd love to have it back in our project.

[DanySK]

@dmedinag as soon as the refreshVersions plugin will support TOML catalogs, UpGradle will provide Dependabot-like functions for Gradle projects through refreshVersions.

[tbroyer]

@DanySK Dependabot powers GitHub's security alerts, it's not only about having automatic updates through pull requests.

[DanySK]

@tbroyer sure, but if @dmedinag is dropping all of Dependabot functionality, in that way at least he can get updates back. Having dependabot supporting Gradle better would be great for me too (that's why I'm following the discussion here...).

[jurre]

@DanySK Dependabot powers GitHub's security alerts, it's not only about having automatic updates through pull requests.

These are currently actually separate systems, Alerts power security updates, but dependabot is not used to scan for alerts at this time. (We hope to bring those systems implementations together at some point, but we've not been able to prioritize it)

[tbroyer]

@jurre As someone who would like alerts, and doesn't really care about automatic updates, is the code open source and could I contribute to it to add support for those TOML files?