cyclonedx-gradle-plugin icon indicating copy to clipboard operation
cyclonedx-gradle-plugin copied to clipboard

Overlapping outputs issue with Spring 3.3.x and Gradle 8.x when building multiple BOMs

Open ThomGeG opened this issue 6 months ago • 1 comments

Hey all,

I'm leaving this less as a feature request / bug report and more as a "this is something I've encountered that I'm sure other people will also suffer through". There's probably a minor change that can be made to the plugin to prevent future headaches however.

I was recently trying to bump some projects of mine to Gradle 8.x and Spring 3.3.x but wasn't able to because of how CycloneDX handles its outputs. Spring 3.3.x added new features around SBOMs that copy them into your WARs/JARs if you use the CycloneDX plugin. This works fine when you're building just one BOM but when you're building more, like I do, it falls apart:

> Task :cyclonedxBuildBom FAILED

FAILURE: Build failed with an exception.

* What went wrong:
A problem was found with the configuration of task ':cyclonedxBuildBom' (type 'CycloneDxTask').
  - Gradle detected a problem with the following location: 'C:\Users\ThomGeG\git\<my-project>\build\reports'.

    Reason: Task ':processResources' uses this output of task ':cyclonedxBuildBom' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.

    Possible solutions:
      1. Declare task ':cyclonedxBuildBom' as an input of ':processResources'.
      2. Declare an explicit dependency on ':cyclonedxBuildBom' from ':processResources' using Task#dependsOn.
      3. Declare an explicit dependency on ':cyclonedxBuildBom' from ':processResources' using Task#mustRunAfter.

    For more information, please refer to https://docs.gradle.org/8.10/userguide/validation_problems.html#implicit_dependency in the Gradle documentation.

Specifically I've got two CycloneDxTask, cyclonedxBom (the automatically created one) that builds me BOMs for the runtime depenedncies and another, cyclonedxBuildBom, that builds them for the non-runtime dependencies (test scope and build pipeline stuff). The problem is that those tasks have overlapping outputs because CycloneDxTask uses an @OutputDirectory rather than a specific @OutputFile and I hadn't made them distinct folders:

https://github.com/CycloneDX/cyclonedx-gradle-plugin/blob/d137d9bdddcd9ea7573fdc2b83e1c9cb774ab10f/src/main/java/org/cyclonedx/gradle/CycloneDxTask.java#L264-L267

While processResources has an input of just build/reports/application-bom.json, that file actually exists in a directory that's the output of two different tasks. Gradle 8.x tightened how it treats overlapping outputs (this stuff will work fine on Gradle 7.x), spitting the dummy when it finds them because they're bad practice that compromise its caching mechanisms. processResources is being specifically mentioned because the new Spring feature works by reconfiguring processResources to copy the BOM producted by cyclonedxBom to META-INF/sbom so it gets zipped into the JARs/WARs:

https://github.com/spring-projects/spring-boot/blob/3.3.x/spring-boot-project/spring-boot-tools/spring-boot-gradle-plugin/src/main/java/org/springframework/boot/gradle/plugin/CycloneDxPluginAction.java#L71-L76

cyclonedxBuildBom is problematic because it hasn't been wired to processResources but cyclonedxBom is fine because it has.

There's two ways I see to address this:

Rework how the CycloneDxTask defines its outputs

There's an example of how to handle tasks that create files in the Gradle docs. If the CycloneDX plugin were to follow it that would mean:

  • Rather than having @OutputDirectory destination we'd instead have @OutputFile output.
  • There'd be three inputs for controlling the output file, outputName, outputDirectory, outputFormat.
  • Those three inputs would be used to define said @OutputFile (<outputDirectory>/<outputName>.<outputFormat>).
  • Each task would only produce a single file, so no more outputFormat = 'all'. If you need multiple formats, you'd need mutliple tasks.

I'm not sure how I feel about this as a general solution. outputFormat = 'all' seems very convenient for getting both XML and JSON BOMs without needing to scrape all the data again. This would also be a breaking change because of all the input/output changes and would mean going to 2.x.

It would however seem to definitively solve the overlapping outputs problem as there's no more directories to have overlap.

Reconfigure cyclonedxBuildBom and cyclonedxBom to have their own distinct output folders

This is much more straightforward and what I'll be doing, at least in the short term. That'd mean:

  • Reconfiguring cyclonedxBom to output to something like build/reports/cyclonedx/application
  • Reconfiguring cyclonedxBuildBom output to something like build/reports/cyclonedx/build
  • Registering a Copy task that copies the BOMs from both to a shared directory, build/reports/boms, for things like Jenkins.

It seems like if this is the desired solution then the default output location should be changed to something like build/reports/boms or build/reports/cyclonedx w/ a note in the README.md about registering your own tasks.


Additional context I started a thread on the Gradle Community Slack here that also has some details about the problem and feedback from the Gradle maintainers.

There's also some stuff here about why overlapping outputs are bad and an open issue in GitHub where they plan to deprecate them entirely.

ThomGeG avatar Aug 19 '24 00:08 ThomGeG