[Gradle] Run dependencies task in parallel for larger multi-project builds.

[ajmalab]

Hey team! Been away for a while, but very glad to see that the tool has come really far these past few months!

I've been doing some experiments on speeding up Gradle project sbom generation. This is currently extremely slow for multi-project builds, as we run ./gradlew dependencies on each sub-project sequentially. While this might work ok on smaller projects, it is quite slow with larger projects (a personal test project with 100+ sub-projects took around ~15m).

The dependencies task is a parallelisable task in Gradle, and hence, can run on multiple sub-projects in parallel. The way to do this would be to concatenate all the :sub-project:dependencies calls into a single gradle command, and execute it with the --parallel and --max-workers options set. Eg: ./gradlew dependencies :sub-project1:dependencies :sub-project2:dependencies .... --parallel --max-workers=4. In my personal tests, this cut down the time from about ~20 minutes down to ~1 minute.

Implementing this would require changing the parseGradleDep function (or adding a separate one for parallel execution), since the command would return a single stream of output with multiple "Project " and corresponding dependencies (see: https://github.com/ajmalab/dependency-diff-check/blob/main/dependencies.txt). If you think this would be a good improvement, would be happy to take a stab at implementing this.

[prabhu]

@ajmalab, welcome back! This is a cool suggestion! Please get a branch going so that we can collaborate and get it working.

[prabhu]

@ajmalab, how are you doing with this ticket? Do you need any help?

[ajmalab]

Hey @prabhu, unfortunately no progress on this yet, been quite busy with some other stuff, so had trouble prioritising this. Hopefully should be able to devote time to this this week.