feat(builder): Add Maven Daemon support

[astefanutti]

This PR aims at exploring support for https://github.com/mvndaemon/mvnd as a build strategy.

Fixes #1784.

TODO:

• [x] Fix consecutive build requests issue
• [ ] Add the mvnd build strategy
• [ ] Fix logging configuration

Release Note

feat(builder): Add Maven Daemon support

[astefanutti]

While the first build request passes:

{"level":"info","ts":1639501717.1190414,"logger":"camel-k.maven","msg":"executing: mvnd --no-transfer-progress -Dstyle.color=never -Dmaven.repo.local=/tmp/artifacts/m2 --settings /tmp/kit-c6scv571rsj03ggdc7m0-118741952/maven/settings.xml package","MAVEN_OPTS":"","PWD":"/tmp/kit-c6scv571rsj03ggdc7m0-118741952/maven"}
{"level":"info","ts":1639501717.1405191,"logger":"camel-k.controller.integrationkit","msg":"Build running","request-namespace":"camel-k","request-name":"kit-c6scv571rsj03ggdc7m0","api-version":"camel.apache.org/v1","kind":"IntegrationKit","ns":"camel-k","name":"kit-c6scv571rsj03ggdc7m0"}
{"level":"info","ts":1639501719.310807,"logger":"camel-k.maven.build","msg":"[INFO] Processing build on daemon 4162007d"}
{"level":"info","ts":1639501719.3461616,"logger":"camel-k.maven.build","msg":"[INFO] Scanning for projects..."}
{"level":"info","ts":1639501722.074389,"logger":"camel-k.controller.build","msg":"Reconciling Build","request-namespace":"camel-k","request-name":"kit-c6scv571rsj03ggdc7m0"}
{"level":"info","ts":1639501722.0744765,"logger":"camel-k.controller.build","msg":"Invoking action monitor-routine","request-namespace":"camel-k","request-name":"kit-c6scv571rsj03ggdc7m0","api-version":"camel.apache.org/v1","kind":"Build","ns":"camel-k","name":"kit-c6scv571rsj03ggdc7m0"}
{"level":"info","ts":1639501722.5713582,"logger":"camel-k.maven.build","msg":"[INFO] BuildTimeEventSpy is registered."}
{"level":"info","ts":1639501722.5806704,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501722.5807147,"logger":"camel-k.maven.build","msg":"[INFO] Using the SmartBuilder implementation with a thread count of 5"}
{"level":"info","ts":1639501722.581693,"logger":"camel-k.maven.build","msg":"[INFO] Task segments : [package]"}
{"level":"info","ts":1639501722.581737,"logger":"camel-k.maven.build","msg":"[INFO] Build maximum degree of concurrency is 5"}
{"level":"info","ts":1639501722.5817523,"logger":"camel-k.maven.build","msg":"[INFO] Total number of projects is 1"}
{"level":"info","ts":1639501722.6017418,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501722.6018167,"logger":"camel-k.maven.build","msg":"[INFO] ---------< org.apache.camel.k.integration:camel-k-integration >---------"}
{"level":"info","ts":1639501722.6018343,"logger":"camel-k.maven.build","msg":"[INFO] Building camel-k-integration 1.8.0-SNAPSHOT"}
{"level":"info","ts":1639501722.6018524,"logger":"camel-k.maven.build","msg":"[INFO] --------------------------------[ jar ]---------------------------------"}
{"level":"info","ts":1639501730.6731327,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501730.6739473,"logger":"camel-k.maven.build","msg":"[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ camel-k-integration ---"}
{"level":"info","ts":1639501731.4879925,"logger":"camel-k.maven.build","msg":"[INFO] Using 'UTF-8' encoding to copy filtered resources."}
{"level":"info","ts":1639501731.4944088,"logger":"camel-k.maven.build","msg":"[INFO] Copying 1 resource"}
{"level":"info","ts":1639501731.5078084,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501731.5078814,"logger":"camel-k.maven.build","msg":"[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ camel-k-integration ---"}
{"level":"info","ts":1639501732.8577766,"logger":"camel-k.maven.build","msg":"[INFO] No sources to compile"}
{"level":"info","ts":1639501732.8585966,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501732.8587456,"logger":"camel-k.maven.build","msg":"[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ camel-k-integration ---"}
{"level":"info","ts":1639501732.8627934,"logger":"camel-k.maven.build","msg":"[INFO] Using 'UTF-8' encoding to copy filtered resources."}
{"level":"info","ts":1639501732.8639426,"logger":"camel-k.maven.build","msg":"[INFO] skip non existing resourceDirectory /tmp/kit-c6scv571rsj03ggdc7m0-118741952/maven/src/test/resources"}
{"level":"info","ts":1639501732.8647652,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501732.8648124,"logger":"camel-k.maven.build","msg":"[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ camel-k-integration ---"}
{"level":"info","ts":1639501732.8696568,"logger":"camel-k.maven.build","msg":"[INFO] No sources to compile"}
{"level":"info","ts":1639501732.86972,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501732.869747,"logger":"camel-k.maven.build","msg":"[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ camel-k-integration ---"}
{"level":"info","ts":1639501733.2105863,"logger":"camel-k.maven.build","msg":"[INFO] No tests to run."}
{"level":"info","ts":1639501733.2191017,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501733.2191536,"logger":"camel-k.maven.build","msg":"[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ camel-k-integration ---"}
{"level":"info","ts":1639501733.5490773,"logger":"camel-k.maven.build","msg":"[INFO] Building jar: /tmp/kit-c6scv571rsj03ggdc7m0-118741952/maven/target/camel-k-integration-1.8.0-SNAPSHOT.jar"}
{"level":"info","ts":1639501733.5680315,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501733.5680842,"logger":"camel-k.maven.build","msg":"[INFO] --- quarkus-maven-plugin:2.4.0.Final:build (default) @ camel-k-integration ---"}
{"level":"info","ts":1639501740.7192912,"logger":"camel-k.maven.build","msg":"[INFO] [org.jboss.threads] JBoss Threads version 3.4.2.Final"}
{"level":"info","ts":1639501742.1674368,"logger":"camel-k.maven.build","msg":"[INFO] [io.quarkus.deployment.QuarkusAugmentor] Quarkus augmentation completed in 2191ms"}
{"level":"info","ts":1639501742.1917005,"logger":"camel-k.maven.build","msg":"[INFO] Segment walltime 19 s, segment projects service time 19 s, effective/maximum degree of concurrency 1.00/5"}
{"level":"info","ts":1639501742.1985629,"logger":"camel-k.maven.build","msg":"[INFO] ------------------------------------------------------------------------"}
{"level":"info","ts":1639501742.1986232,"logger":"camel-k.maven.build","msg":"[INFO] BUILD SUCCESS"}
{"level":"info","ts":1639501742.19864,"logger":"camel-k.maven.build","msg":"[INFO] ------------------------------------------------------------------------"}
{"level":"info","ts":1639501742.1986544,"logger":"camel-k.maven.build","msg":"[INFO] Total time:  22.888 s (Wall Clock)"}
{"level":"info","ts":1639501742.2030387,"logger":"camel-k.maven.build","msg":"[INFO] Finished at: 2021-12-14T17:09:02Z"}
{"level":"info","ts":1639501742.2075596,"logger":"camel-k.maven.build","msg":"[INFO] ------------------------------------------------------------------------"}

During the build, the daemon is busy as expected:

$ kubectl exec -it camel-k-operator-6d544f6585-2dlrp -- mvnd --status
ID             PID                   Address   Status    RSS            Last activity  Java home
4162007d       42     inet:/127.0.0.1:37113     Busy   152m  2021-12-14T17:08:39.069  /opt/mandrel

However, any subsequent build request fails:

{"level":"info","ts":1639501775.790641,"logger":"camel-k.maven","msg":"executing: mvnd --no-transfer-progress -Dstyle.color=never -Dmaven.repo.local=/tmp/artifacts/m2 --settings /tmp/kit-c6scvjv1rsj03ggdc7mg-1605060085/maven/settings.xml package","MAVEN_OPTS":"","PWD":"/tmp/kit-c6scvjv1rsj03ggdc7mg-1605060085/maven"}
{"level":"info","ts":1639501775.825956,"logger":"camel-k.maven.build","msg":"[INFO] Processing build on daemon 4162007d"}
{"level":"info","ts":1639501775.8260071,"logger":"camel-k.maven.build","msg":"[INFO] Scanning for projects..."}
{"level":"info","ts":1639501775.8315053,"logger":"camel-k.maven.build","msg":"[INFO] BuildTimeEventSpy is registered."}
{"level":"info","ts":1639501775.8315558,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501775.8315768,"logger":"camel-k.maven.build","msg":"[INFO] Using the SmartBuilder implementation with a thread count of 5"}
{"level":"info","ts":1639501775.8319685,"logger":"camel-k.maven.build","msg":"[INFO] Task segments : [package]"}
{"level":"info","ts":1639501775.8320405,"logger":"camel-k.maven.build","msg":"[INFO] Build maximum degree of concurrency is 5"}
{"level":"info","ts":1639501775.832054,"logger":"camel-k.maven.build","msg":"[INFO] Total number of projects is 1"}
{"level":"info","ts":1639501775.8382747,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501775.8383217,"logger":"camel-k.maven.build","msg":"[INFO] ---------< org.apache.camel.k.integration:camel-k-integration >---------"}
{"level":"info","ts":1639501775.8383431,"logger":"camel-k.maven.build","msg":"[INFO] Building camel-k-integration 1.8.0-SNAPSHOT"}
{"level":"info","ts":1639501775.838399,"logger":"camel-k.maven.build","msg":"[INFO] --------------------------------[ jar ]---------------------------------"}
{"level":"info","ts":1639501776.3599691,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501776.3600264,"logger":"camel-k.maven.build","msg":"[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ camel-k-integration ---"}
{"level":"info","ts":1639501776.3643596,"logger":"camel-k.maven.build","msg":"[INFO] Using 'UTF-8' encoding to copy filtered resources."}
{"level":"info","ts":1639501776.365225,"logger":"camel-k.maven.build","msg":"[INFO] Copying 1 resource"}
{"level":"info","ts":1639501776.3664856,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501776.3665676,"logger":"camel-k.maven.build","msg":"[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ camel-k-integration ---"}
{"level":"info","ts":1639501776.370944,"logger":"camel-k.maven.build","msg":"[INFO] No sources to compile"}
{"level":"info","ts":1639501776.371192,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501776.3714032,"logger":"camel-k.maven.build","msg":"[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ camel-k-integration ---"}
{"level":"info","ts":1639501776.3733745,"logger":"camel-k.maven.build","msg":"[INFO] Using 'UTF-8' encoding to copy filtered resources."}
{"level":"info","ts":1639501776.3734963,"logger":"camel-k.maven.build","msg":"[INFO] skip non existing resourceDirectory /tmp/kit-c6scvjv1rsj03ggdc7mg-1605060085/maven/src/test/resources"}
{"level":"info","ts":1639501776.373666,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501776.3742728,"logger":"camel-k.maven.build","msg":"[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ camel-k-integration ---"}
{"level":"info","ts":1639501776.3807945,"logger":"camel-k.maven.build","msg":"[INFO] No sources to compile"}
{"level":"info","ts":1639501776.3815188,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501776.3815703,"logger":"camel-k.maven.build","msg":"[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ camel-k-integration ---"}
{"level":"info","ts":1639501776.386186,"logger":"camel-k.maven.build","msg":"[INFO] No tests to run."}
{"level":"info","ts":1639501776.386649,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501776.3869555,"logger":"camel-k.maven.build","msg":"[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ camel-k-integration ---"}
{"level":"info","ts":1639501776.3941643,"logger":"camel-k.maven.build","msg":"[INFO] Building jar: /tmp/kit-c6scvjv1rsj03ggdc7mg-1605060085/maven/target/camel-k-integration-1.8.0-SNAPSHOT.jar"}
{"level":"info","ts":1639501776.3987367,"logger":"camel-k.maven.build","msg":"[INFO] "}
{"level":"info","ts":1639501776.3988311,"logger":"camel-k.maven.build","msg":"[INFO] --- quarkus-maven-plugin:2.4.0.Final:build (default) @ camel-k-integration ---"}
{"level":"info","ts":1639501776.4104228,"logger":"camel-k.maven.build","msg":"[INFO] ------------------------------------------------------------------------"}
{"level":"info","ts":1639501776.4104898,"logger":"camel-k.maven.build","msg":"[INFO] BUILD FAILURE"}
{"level":"info","ts":1639501776.41051,"logger":"camel-k.maven.build","msg":"[INFO] ------------------------------------------------------------------------"}
{"level":"info","ts":1639501776.410534,"logger":"camel-k.maven.build","msg":"[INFO] Total time:  0.588 s (Wall Clock)"}
{"level":"info","ts":1639501776.410554,"logger":"camel-k.maven.build","msg":"[INFO] Finished at: 2021-12-14T17:09:36Z"}
{"level":"info","ts":1639501776.4109573,"logger":"camel-k.maven.build","msg":"[INFO] ------------------------------------------------------------------------"}
{"level":"info","ts":1639501776.4157386,"logger":"camel-k.maven.build","msg":"[ERROR] Failed to execute goal io.quarkus:quarkus-maven-plugin:2.4.0.Final:build (default) on project camel-k-integration: Failed to build quarkus application: Error while reading file as JAR: /tmp/kit-c6scv571rsj03ggdc7m0-118741952/maven/target/camel-k-integration-1.8.0-SNAPSHOT.jar -> [Help 1]"}

It seems the daemon tries to access a JAR file built during the previous build. However, this file does not exist anymore, as the project has been deleted. That seems surprising a build tries to access files from another build.

@gnodet @ppalaga would you be kind enough to shed your expertise on this issue. It's probably a mis-configuration in the way builds are run by Camel K, but it'll probably take me ages to figure it out without your help 😀.

[ppalaga]

quarkus-maven-plugin is known to have issues on mvnd. Could you please add -e to the build command to see where exactly the issue happens? My wild guess is that this is a variant of https://github.com/quarkusio/quarkus/issues/21718 whose root cause is explained in https://github.com/quarkusio/quarkus/pull/21719#discussion_r763950863 So you may want try to add <extensions>true</extensions> to the quarkus-maven-plugin declaration in the generated pom.xml and see whether it helps. Except for that, there is an option to instruct mvnd to evict some plugin's class loaders after a mvnd execution via mvnd.pluginRealmEvictPattern, see https://github.com/gnodet/mvnd/commit/9ed8647014c311e5711bb8c40afa7f6abc4ea5f4#diff-e65744bdec30df4a8c9692d35f97c28fef1e81af93a54a3d0dbf5f0a8e8abd88R246

I am not sure myself about the syntax, I have filed https://github.com/mvndaemon/mvnd/issues/533 to clear with @gnodet

[astefanutti]

@ppalaga thanks, I confirm adding <extensions>true</extensions> to the quarkus-maven-plugin declaration fixes the issue!

[oscerd]

I think we need to resume the work on this.

[orpiske]

I think we need to resume the work on this.

+1. I think this would be great for Camel K.

[astefanutti]

I think we need to resume the work on this.

+1. I think this would be great for Camel K.

It's one of the topics proposed in the Outreachy internships: https://www.outreachy.org/apply/project-selection/#apache

[github-actions[bot]]

This PR has been automatically marked as stale due to 90 days of inactivity. It will be closed if no further activity occurs within 15 days. If you think that’s incorrect or the issue should never stale, please simply write any comment. Thanks for your contributions!

[whtssub]

The PR is in progress