maven-build-cache-extension
maven-build-cache-extension copied to clipboard
apache
[MBUILDCACHE-78] Maven reactor doesn't work well for a multimodule project when enabling build cache
Hongxin Liang opened MBUILDCACHE-78 and commented
In a multimodule project, as a simplest example: https://github.com/honnix/cache , "mvn compile" with build cache enabled doesn't play well with Maven reactor. A few scenarios I have tested in the provided repro example:
- With a clean environment (no build cache, no "mvn install" have been done)
- Run "mvn compile" and everything works
- Modify (putting a random empty line will suffice) class "B" in module "b"
- Run "mvn compile" again, and now it fails when trying to download snapshot of module "a" from maven central (after failing to find that in my local repo)
- With a clean environment (no build cache)
- Run "mvn install" and everything works
- Modify class "A" in module "a" to rename the method
- Run "mvn compile" for the first time, and it fails to compile due to renaming of the method in class "A"
- Run "mvn compile" for the second time, nothing fails but apparent the code is still wrong; run more "mvn compile" and still nothing fails
- Running "mvn install" fails the build due to compilation error
I'm not entirely sure I am doing things properly and I'm not sure whether the config is correct either (I took the example as https://maven.apache.org/extensions/maven-build-cache-extension/maven-build-cache-config.xml with modification to disable remote cache).
Affects: 1.1.0
Johan Parent commented
Hi,
I confirm. The checksum for the pom seems to be different each run.
Having looked briefly at the code I suppose the effecitve pom representation contains something that "doesn't hash" consistently. A missing hashCode() implementation, but I speculate.
Regards,
Johan
Johan Parent commented
Hello Hongxin Liang ,
I tried on your cache project. Thing seems to work, no? The checksums match and files get restored. Or is their content wrong?
Regards,
Johan
INFO] --------------------------< io.honnix:cache >---------------------------
[INFO] Building cache 1.0-SNAPSHOT [1/3]
[INFO] from pom.xml
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] ----------------------------< io.honnix:a >-----------------------------
[INFO] Building a 1.0-SNAPSHOT [2/3]
[INFO] from a/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=a]
[INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 6, plugins: 4 millis
[INFO] Project inputs calculated in 25 ms. SHA-256 checksum [585351936dd02c1a4d84a3384e83fa66698e678148b14c141a3fb8855e32df6c] calculated in 24 ms.
[INFO] Attempting to restore project io.honnix:a from build cache
[INFO] Local build found by checksum 585351936dd02c1a4d84a3384e83fa66698e678148b14c141a3fb8855e32df6c
[INFO] Found cached build, restoring io.honnix:a from cache by checksum 585351936dd02c1a4d84a3384e83fa66698e678148b14c141a3fb8855e32df6c
[INFO] Skipping plugin execution (cached): resources:resources
[INFO] Skipping plugin execution (cached): compiler:compile
[INFO] Skipping plugin execution (cached): resources:testResources
[INFO] Skipping plugin execution (cached): compiler:testCompile
[INFO] Skipping plugin execution (cached): surefire:test
[INFO]
[INFO] ----------------------------< io.honnix:b >-----------------------------
[INFO] Building b 1.0-SNAPSHOT [3/3]
[INFO] from b/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=b]
[INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 2, plugins: 1 millis
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=a]
[INFO] Project inputs calculated in 6 ms. SHA-256 checksum [4fded1cb646674a216944e06b31754cb5a90c70ea90b8c1acc18f56fd11a6a7b] calculated in 2 ms.
[INFO] Attempting to restore project io.honnix:b from build cache
[INFO] Local build found by checksum 4fded1cb646674a216944e06b31754cb5a90c70ea90b8c1acc18f56fd11a6a7b
[INFO] Found cached build, restoring io.honnix:b from cache by checksum 4fded1cb646674a216944e06b31754cb5a90c70ea90b8c1acc18f56fd11a6a7b
[INFO] Skipping plugin execution (cached): resources:resources
[INFO] Skipping plugin execution (cached): compiler:compile
[INFO] Skipping plugin execution (cached): resources:testResources
[INFO] Skipping plugin execution (cached): compiler:testCompile
[INFO] Skipping plugin execution (cached): surefire:test
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for cache 1.0-SNAPSHOT:
[INFO]
[INFO] cache .............................................. SUCCESS [ 0.005 s]
[INFO] a .................................................. SUCCESS [ 0.279 s]
[INFO] b .................................................. SUCCESS [ 0.019 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.390 s
[INFO] Finished at: 2024-05-17T13:14:50+02:00
[INFO] ------------------------------------------------------------------------
Hongxin Liang commented
Hi parent_johan,
Thank you for looking into this. It has been a while and I don't remember details. Have you tried following the repro steps I put in the description? I can try it out again later asap.
/H
Hongxin Liang commented
I could still produce these two using 1.2.0 version.
For 1, I got this log:
[INFO] --------------------------< io.honnix:cache >---------------------------
[INFO] Building cache 1.0-SNAPSHOT [1/3]
[INFO] from pom.xml
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] ----------------------------< io.honnix:a >-----------------------------
[INFO] Building a 1.0-SNAPSHOT [2/3]
[INFO] from a/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Loading cache configuration from /Users/honnix/Developer/github/cache/.mvn/maven-build-cache-config.xml
[INFO] Using SHA-256 hash algorithm for cache
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=a]
[INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 1, plugins: 1 millis
[INFO] Project inputs calculated in 3 ms. SHA-256 checksum [d94bcab16e7817cbb356a288016b80130c30e87d3c0e96f63347697334c206f5] calculated in 1 ms.
[INFO] Attempting to restore project io.honnix:a from build cache
[INFO] Local build found by checksum d94bcab16e7817cbb356a288016b80130c30e87d3c0e96f63347697334c206f5
[INFO] Found cached build, restoring io.honnix:a from cache by checksum d94bcab16e7817cbb356a288016b80130c30e87d3c0e96f63347697334c206f5
[INFO] Skipping plugin execution (cached): resources:resources
[INFO] Skipping plugin execution (cached): compiler:compile
[INFO]
[INFO] ----------------------------< io.honnix:b >-----------------------------
[INFO] Building b 1.0-SNAPSHOT [3/3]
[INFO] from b/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=b]
[INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 1, plugins: 1 millis
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=a]
[INFO] Project inputs calculated in 2 ms. SHA-256 checksum [8a63d23bd4e2fb7162d1e6ef5d415a8b35527fc13c1b58122e2af25f52d53de0] calculated in 0 ms.
[INFO] Attempting to restore project io.honnix:b from build cache
[INFO] Local build was not found by checksum 8a63d23bd4e2fb7162d1e6ef5d415a8b35527fc13c1b58122e2af25f52d53de0 for io.honnix:b
[INFO] Downloading from repo: ...
[INFO] Downloading from repo: ...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for cache 1.0-SNAPSHOT:
[INFO]
[INFO] cache .............................................. SUCCESS [ 0.000 s]
[INFO] a .................................................. SUCCESS [ 0.012 s]
[INFO] b .................................................. FAILURE [ 16.822 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16.845 s (Wall Clock)
[INFO] Finished at: 2024-05-17T14:34:19+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project b: Could not resolve dependencies for project io.honnix:b:jar:1.0-SNAPSHOT: The following artifacts could not be resolved: io.honnix:a:jar:1.0-SNAPSHOT (absent): Could not find artifact io.honnix:a:jar:1.0-SNAPSHOT in repo (...) -> [Help 1]
For 2, I got this log:
$ mvn compile
[INFO] Scanning for projects...
[INFO] BuildTimeEventSpy is registered.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] cache [pom]
[INFO] a [jar]
[INFO] b [jar]
[INFO]
[INFO] Using the SmartBuilder implementation with a thread count of 11
[INFO]
[INFO] --------------------------< io.honnix:cache >---------------------------
[INFO] Building cache 1.0-SNAPSHOT [1/3]
[INFO] from pom.xml
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] ----------------------------< io.honnix:a >-----------------------------
[INFO] Building a 1.0-SNAPSHOT [2/3]
[INFO] from a/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Loading cache configuration from /Users/honnix/Developer/github/cache/.mvn/maven-build-cache-config.xml
[INFO] Using SHA-256 hash algorithm for cache
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=a]
[INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 2, plugins: 0 millis
[INFO] Project inputs calculated in 2 ms. SHA-256 checksum [c314c8ae60babb75456ed81056b4c11fe52c0b74d713a623cee87361768a672c] calculated in 0 ms.
[INFO] Attempting to restore project io.honnix:a from build cache
[INFO] Local build was not found by checksum c314c8ae60babb75456ed81056b4c11fe52c0b74d713a623cee87361768a672c for io.honnix:a
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ a ---
[INFO] skip non existing resourceDirectory /Users/honnix/Developer/github/cache/a/src/main/resources
[INFO]
[INFO] --- compiler:3.11.0:compile (default-compile) @ a ---
[INFO] Changes detected - recompiling the module! :source
[INFO] Compiling 1 source file with javac [debug release 17] to target/classes
[INFO] Saved Build to local file: /Users/honnix/.m2/build-cache/v1/io.honnix/a/c314c8ae60babb75456ed81056b4c11fe52c0b74d713a623cee87361768a672c/local/buildinfo.xml
[INFO]
[INFO] ----------------------------< io.honnix:b >-----------------------------
[INFO] Building b 1.0-SNAPSHOT [3/3]
[INFO] from b/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=b]
[INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 0, plugins: 1 millis
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=a]
[INFO] Project inputs calculated in 2 ms. SHA-256 checksum [8bffbe36857d63a0897bc4a9b69f2e2beb50c5170043ad7de09a01e2d790da85] calculated in 0 ms.
[INFO] Attempting to restore project io.honnix:b from build cache
[INFO] Local build was not found by checksum 8bffbe36857d63a0897bc4a9b69f2e2beb50c5170043ad7de09a01e2d790da85 for io.honnix:b
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ b ---
[INFO] skip non existing resourceDirectory /Users/honnix/Developer/github/cache/b/src/main/resources
[INFO]
[INFO] --- compiler:3.11.0:compile (default-compile) @ b ---
[INFO] Changes detected - recompiling the module! :dependency
[INFO] Compiling 1 source file with javac [debug release 17] to target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /Users/honnix/Developer/github/cache/b/src/main/java/io/honnix/B.java:[7,13] cannot find symbol
symbol: method a()
location: class io.honnix.A
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for cache 1.0-SNAPSHOT:
[INFO]
[INFO] cache .............................................. SUCCESS [ 0.001 s]
[INFO] a .................................................. SUCCESS [ 0.039 s]
[INFO] b .................................................. FAILURE [ 0.025 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.075 s (Wall Clock)
[INFO] Finished at: 2024-05-17T14:39:43+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.11.0:compile (default-compile) on project b: Compilation failure
[ERROR] /Users/honnix/Developer/github/cache/b/src/main/java/io/honnix/B.java:[7,13] cannot find symbol
[ERROR] symbol: method a()
[ERROR] location: class io.honnix.A
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <args> -r
$ mvn compile
[INFO] Scanning for projects...
[INFO] BuildTimeEventSpy is registered.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] cache [pom]
[INFO] a [jar]
[INFO] b [jar]
[INFO]
[INFO] Using the SmartBuilder implementation with a thread count of 11
[INFO]
[INFO] --------------------------< io.honnix:cache >---------------------------
[INFO] Building cache 1.0-SNAPSHOT [1/3]
[INFO] from pom.xml
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] ----------------------------< io.honnix:a >-----------------------------
[INFO] Building a 1.0-SNAPSHOT [2/3]
[INFO] from a/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Loading cache configuration from /Users/honnix/Developer/github/cache/.mvn/maven-build-cache-config.xml
[INFO] Using SHA-256 hash algorithm for cache
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=a]
[INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 1, plugins: 0 millis
[INFO] Project inputs calculated in 2 ms. SHA-256 checksum [c314c8ae60babb75456ed81056b4c11fe52c0b74d713a623cee87361768a672c] calculated in 1 ms.
[INFO] Attempting to restore project io.honnix:a from build cache
[INFO] Local build found by checksum c314c8ae60babb75456ed81056b4c11fe52c0b74d713a623cee87361768a672c
[INFO] Found cached build, restoring io.honnix:a from cache by checksum c314c8ae60babb75456ed81056b4c11fe52c0b74d713a623cee87361768a672c
[INFO] Skipping plugin execution (cached): resources:resources
[INFO] Skipping plugin execution (cached): compiler:compile
[INFO]
[INFO] ----------------------------< io.honnix:b >-----------------------------
[INFO] Building b 1.0-SNAPSHOT [3/3]
[INFO] from b/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=b]
[INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 1, plugins: 0 millis
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=a]
[INFO] Project inputs calculated in 2 ms. SHA-256 checksum [8bffbe36857d63a0897bc4a9b69f2e2beb50c5170043ad7de09a01e2d790da85] calculated in 1 ms.
[INFO] Attempting to restore project io.honnix:b from build cache
[INFO] Local build was not found by checksum 8bffbe36857d63a0897bc4a9b69f2e2beb50c5170043ad7de09a01e2d790da85 for io.honnix:b
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ b ---
[INFO] skip non existing resourceDirectory /Users/honnix/Developer/github/cache/b/src/main/resources
[INFO]
[INFO] --- compiler:3.11.0:compile (default-compile) @ b ---
[INFO] Changes detected - recompiling the module! :source
[INFO] Compiling 1 source file with javac [debug release 17] to target/classes
[INFO] Saved Build to local file: /Users/honnix/.m2/build-cache/v1/io.honnix/b/8bffbe36857d63a0897bc4a9b69f2e2beb50c5170043ad7de09a01e2d790da85/local/buildinfo.xml
[INFO] ------------------------------------------------------------------------
[INFO] Average project wall time: 0,00s
[INFO] Total concurrency: 9 %
[INFO] Bottleneck projects that decrease concurrency: (run build with -Dsmartbuilder.profiling=true for further details)
[INFO] - io.honnix:b:1.0-SNAPSHOT
[INFO] - io.honnix:a:1.0-SNAPSHOT
[INFO] - io.honnix:cache:1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for cache 1.0-SNAPSHOT:
[INFO]
[INFO] cache .............................................. SUCCESS [ 0.000 s]
[INFO] a .................................................. SUCCESS [ 0.008 s]
[INFO] b .................................................. SUCCESS [ 0.037 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.054 s (Wall Clock)
[INFO] Finished at: 2024-05-17T14:39:51+02:00
[INFO] ------------------------------------------------------------------------
In general I have a feeling it doesn't play well with maven reactor.
this is actually per design see documentation https://github.com/apache/maven-build-cache-extension/blob/21d9bef261c2a6497ddab2911e998f6aa6bbcec3/src/site/markdown/usage.md?plain=1#L20
