Snapshots being downloaded before modules are built

[coheigea]

Hi,

In Apache WSS4J we have configured the plugin like:

<plugin>
                    <groupId>org.cyclonedx</groupId>
                    <artifactId>cyclonedx-maven-plugin</artifactId>
                    <version>2.7.9</version>
                    <executions>
                        <execution>
                            <id>make-bom</id>
                            <phase>package</phase>
                            <goals>
                                <goal>makeAggregateBom</goal>
                            </goals>
                        </execution>
                    </executions>
                    <configuration>
                        <outputName>${project.artifactId}-${project.version}-bom</outputName>
                    </configuration>
                </plugin>

https://github.com/apache/ws-wss4j/blob/17f33f0cf0680696051117098881df036bd47653/pom.xml#L333C17-L349C26

However on a normal build, it is downloading SNAPSHOTs before it even builds the modules. Also reported for Apache CXF which uses the same configuration here https://github.com/apache/cxf/pull/1480#discussion_r1365443501

[nscuro]

I'm not too deep into how the plugin works, but I also don't quite understand the problem here. Could you elaborate a bit more @coheigea?

IIRC, the plugin will be invoked for every module in the Maven reactor, so seeing an execution per module is expected. In order to determine the libraries used, and how they're related, the plugin needs to resolve all dependencies first. If a module depends on SNAPSHOT libraries, then those would be downloaded (or resolved from ~/.m2).

[reta]

IIRC, the plugin will be invoked for every module in the Maven reactor, so seeing an execution per module is expected. In order to determine the libraries used, and how they're related, the plugin needs to resolve all dependencies first. If a module depends on SNAPSHOT libraries, then those would be downloaded (or resolved from ~/.m2).

@nscuro I will try to share more details here (using https://github.com/apache/cxf/pull/1480#discussion_r1365443501) as an example. Let say we want to have a build of the Apache CXF from main branch (which currently has version4.0.4-SNAPSHOT version) but we bumping it to 4.0.5-SNAPSHOT (so we have a brand new version). The problem is that cyclonedx-maven-plugin tries to fetch 4.0.5-SNAPSHOT of the Apache CXF before they are built actually.

[INFO] -------------------------< org.apache.cxf:cxf >-------------------------
[INFO] Building Apache CXF 4.0.5-SNAPSHOT                               [1/188]
[INFO]   from pom.xml
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- enforcer:3.4.1:enforce (enforce-maven) @ cxf ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.version.RequireJavaVersion passed
[INFO] Rule 1: org.apache.maven.enforcer.rules.version.RequireMavenVersion passed
[INFO] Rule 2: org.apache.maven.enforcer.rules.dependency.BannedDependencies passed
[INFO]
[INFO] --- cyclonedx:2.7.10:makeAggregateBom (make-bom) @ cxf ---
[INFO] CycloneDX: Resolving Aggregated Dependencies
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-core/4.0.5-SNAPSHOT/maven-metadata.xml
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-core/4.0.5-SNAPSHOT/cxf-core-4.0.5-SNAPSHOT.jar
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-rt-wsdl/4.0.5-SNAPSHOT/maven-metadata.xml
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-rt-wsdl/4.0.5-SNAPSHOT/cxf-rt-wsdl-4.0.5-SNAPSHOT.jar
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-rt-databinding-jaxb/4.0.5-SNAPSHOT/maven-metadata.xml
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-rt-databinding-jaxb/4.0.5-SNAPSHOT/cxf-rt-databinding-jaxb-4.0.5-SNAPSHOT.jar
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-tools-common/4.0.5-SNAPSHOT/maven-metadata.xml
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-rt-bindings-soap/4.0.5-SNAPSHOT/maven-metadata.xml
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-tools-common/4.0.5-SNAPSHOT/cxf-tools-common-4.0.5-SNAPSHOT.jar
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-rt-bindings-soap/4.0.5-SNAPSHOT/cxf-rt-bindings-soap-4.0.5-SNAPSHOT.jar
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-tools-common/4.0.5-SNAPSHOT/cxf-tools-common-4.0.5-SNAPSHOT-tests.jar
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-tools-validator/4.0.5-SNAPSHOT/maven-metadata.xml
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-tools-validator/4.0.5-SNAPSHOT/cxf-tools-validator-4.0.5-SNAPSHOT.jar
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-tools-wsdlto-core/4.0.5-SNAPSHOT/maven-metadata.xml
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-tools-wsdlto-core/4.0.5-SNAPSHOT/cxf-tools-wsdlto-core-4.0.5-SNAPSHOT.jar
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-tools-wsdlto-frontend-jaxws/4.0.5-SNAPSHOT/maven-metadata.xml
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-rt-javascript/4.0.5-SNAPSHOT/maven-metadata.xml
...

To generalize the problem a bit, the plugin tries to fetch the dependencies of the modules before building these modules (fetching the external snapshots are totally fine). Hope it is helpful

[ppkarwasz]

I can confirm this issue. When we bind the makeAggregateBom goal to the lifecycle:

1. The root module is the first module of the reactor,
2. The CycloneDX plugin has therefore no way to access the project's JARs, so it downloads the previous snapshot and uses its hashes in the SBOM.

Therefore:

• for snapshots, the hashes of the artifacts are wrong (they refer to the previous snapshot),
• for releases, the hashes of the artifacts are missing.

[prabhu]

@ppkarwasz are you talking about the hashes for the metadata.component (parent component) being incorrect? Have you seen issues with the components list?

[ppkarwasz]

@prabhu,

Looking at this again, metadata.component doesn't have any hashes and neither does any of the component elements from a Maven multi-module build. See log4j-bom-2.23.1-cyclonedx.xml for example: neither log4j-bom, nor any of the sub-modules (log4j-api, log4j-core) have a hashes key.

If there were hashes in those components, they would certainly be wrong, because the goal makeAggregateBom for log4j-bom is executed before any of the jar tasks that create the artifacts.

[prabhu]

@ppkarwasz Thank you. Best to track this as a separate issue, since the hashes are missed for key components. Is there an easy project to reproduce this?

[garydgregory]

This is a problem for Apache Log4j as well.