Idea proposal: nixpkgs Gradle tooling

[rafaelsgirao]

Hi! I'm Rafael Girão from Portugal and I'm finishing a Bsc. in Computer Science & Engineering @ Técnico Lisboa this spring.

I will be applying to NixOS's Summer of Code this year. While I'll be applying to an existing idea, I would also like to propose an idea of my own:

Proposed idea: "nixpkgs Gradle tooling"

Problem summary

From my own mild research of nixpkgs, there seems to be no standard way of packaging Gradle applications as of today. Each package author seems to choose their own approach to building Gradle-dependant software. Some examples:

• Some use hard-to-read perl scripts to mavenify Gradle dependencies, which then requires mangling the source's build files to work: openjfx, armitage , signald, key
• Some projects which just have dependencies that use Gradle simply ignore the problem by fetching pre-build JARs: Aeron-cpp

The problem with these methods is that they perform an entire build inside a fixed-output-derivation just to achieve the side effect of getting all project's dependencies cached locally. As a result, packages have weak reproducibility guarantees and computational time is spent building the project twice (once for fetching dependencies and once for truly building it).

Proposal

Create tooling in nixpkgs to standardize Gradle packaging methods, similar to the tooling available for Maven projects:

• fetch a project's Gradle dependencies, e.g fetchGradleDeps
  • To avoid building the whole project inside a fixed-output-derivation, I propose using Gradle's new dependency verification feature, which fetches all project dependencies as a side-effect. I hope this approach also translates to stronger reproducibility guarantees.
• create a stdenv-based utility for building Gradle packages, e.g buildGradlePackage.
• (?) Convert existing Gradle software packages to use this new tooling.

Expected effort

Since I think this idea is very similar to nixpkgs pnpm tooling, I believe it's a medium (175h) with a rating of hard.

---

Please let me know your thoughts on this idea.

Thanks!

[dblsaiko]

Expected effort

Since I think this idea is very similar to nixpkgs pnpm tooling, I believe it's a medium (175h) with a rating of hard.

I really hope you know what you're signing yourself up for here.

From my experience (that is, having written a gentoo eclass for npm packages – gentoo packages similarly can't connect to the network during build – and having had the misfortune of having to deal with gradle for years), gradle is much more demented than npm, not only because external plugins that execute arbitrary code are common, and I would expect it to be a lot more difficult as a result.

(Then again, I haven't looked too closely at it in a while. Things might have changed that makes this feasible to do with similar amount of effort.)

[TomaSajt]

Here's a PR also tackling the gradle problem: https://github.com/NixOS/nixpkgs/pull/272380

[chayleaf]

I have looked into everything Gradle has to offer (including dependency verification), and, short of writing a Gradle plugin, there are no good ways of achieving this cleanly. This is exacerbated by the fact many Gradle projects download non-Maven dependencies (like native libs) in non-declarative fashion.

That's why my PR goes for an "unclean" approach of just MITMing Gradle. It works, and what's left is sorting out some security concerns and edge cases.

It would perhaps be nice if someone developed a clean solution instead, using a Gradle plugin, however that would in some aspects be inferior to the MITM approach, as it would only allow automatically fetching Maven dependencies.

I'm of course willing to answer any Gradle-related questions, but I can't help with plugin development as I'm not a Java dev.