[MINSTALL-202] Version property is not expanded for destination path

[jira-importer]

Jimisola Laursen opened MINSTALL-202 and commented

I recently contributed the goal versions:use-dynamic-version-from-scm to MojoHaus Versions Maven plugin with the help of sjaranowski .

The goal is to allow dynamic versioning using Maven CI Friendly Versions by setting a property (revision), i.e. 

<version>${revision}</version> 

However, it turns out that 

./mvnw org.codehaus.mojo:versions-maven-plugin:2.17.1:use-dynamic-version-from-scm install -useVersion=0.0.0

does not give the correct destination path but rather:

/home/u30576/.m2/repository/se/lfv/lips3/microservices/ms001/${revision}/ms001-${revision}-spring-boot.jar

as seen below.

 

u30576@DCL0004:~/dev/clones/sysdev/lips3/microservices/ms001 (main)$./mvnw org.codehaus.mojo:versions-maven-plugin:2.17.1:use-dynamic-version-from-scm install -useVersion=0.0.0
[INFO] ------------------< se.lfv.lips3.microservices:ms001 >------------------
[INFO] Building ms001 ${revision}
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- versions:2.17.1:use-dynamic-version-from-scm (default-cli) @ ms001 ---
[INFO] Property 'revision' set to: 0.0.0
...
[INFO] --- jar:3.4.2:jar (default-jar) @ ms001 ---
[INFO] Building jar: /home/u30576/dev/clones/sysdev/lips3/microservices/ms001/target/ms001-0.0.0.jar
[INFO] 
[INFO] --- javadoc:3.8.0:jar (se.lfv.maven.tiles_lfv-development-environment-tile_0.2.11__attach-javadocs) @ ms001 ---
[INFO] Configuration changed, re-generating javadoc.
[INFO] Building jar: /home/u30576/dev/clones/sysdev/lips3/microservices/ms001/target/ms001-0.0.0-javadoc.jar
[INFO] 
[INFO] --- spring-boot:3.3.2:repackage (se.lfv.maven.tiles_spring-boot-application-tile_0.4.1__repackage) @ ms001 ---
[INFO] Attaching repackaged archive /home/u30576/dev/clones/sysdev/lips3/microservices/ms001/target/ms001-0.0.0-spring-boot.jar with classifier spring-boot
[INFO] 
[INFO] 
[INFO] --- install:3.1.2:install (default-install) @ ms001 ---
[INFO] Installing /home/u30576/dev/clones/sysdev/lips3/microservices/ms001/pom.xml to /home/u30576/.m2/repository/se/lfv/lips3/microservices/ms001/${revision}/ms001-${revision}.pom
[INFO] Installing /home/u30576/dev/clones/sysdev/lips3/microservices/ms001/target/ms001-0.0.0.jar to /home/u30576/.m2/repository/se/lfv/lips3/microservices/ms001/${revision}/ms001-${revision}.jar
[INFO] Installing /home/u30576/dev/clones/sysdev/lips3/microservices/ms001/target/ms001-0.0.0-javadoc.jar to /home/u30576/.m2/repository/se/lfv/lips3/microservices/ms001/${revision}/ms001-${revision}-javadoc.jar
[INFO] Installing /home/u30576/dev/clones/sysdev/lips3/microservices/ms001/target/ms001-0.0.0-spring-boot.jar to /home/u30576/.m2/repository/se/lfv/lips3/microservices/ms001/${revision}/ms001-${revision}-spring-boot.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  6.410 s
[INFO] Finished at: 2024-07-30T13:49:32+02:00
[INFO] ------------------------------------------------------------------------
 

 

I'm aware that Maven outputs "[INFO] Building ms001 ${revision}" initially (before the plugin has run) but I reckon that initialize is the correct phase to execute the plugin in.

Build Info:

Apache Maven 3.9.8 (36645f6c9b5079805ea5009217e36f2cffd34256) Maven home: /home/uXXXXX/.m2/wrapper/dists/apache-maven-3.9.8/d33422ce Java version: 21.0.1, vendor: Eclipse Adoptium, runtime: /opt/dev/sdkman/candidates/java/21.0.1-tem Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "6.5.0-45-generic", arch: "amd64", family: "unix"

 

---

Issue Links:

• MNG-8211 Maven should fail builds that use CI Friendly versions but have no values set

1 votes, 3 watchers

[jira-importer]

Jimisola Laursen commented

Tamas Cservenak My apologies for pinging you like this. I'm just not sure how to eyes on this issue. This bug is a blocker for a feature added by me and sjaranowski MojoHaus Versions Maven plugin.

[jira-importer]

Jimisola Laursen commented

Thank you VERY much. Looking forward to being able to use dynamic versioning for our Maven projects. There could of course be other core Maven plugins with this issue, but hopefully not.

Any idea on when 3.1.3 will be released? I'll monitor the Maven mailing list for VOTE and ANN.

Thanks once again.

[jira-importer]

Tamas Cservenak commented

Re "when", this is last issue scheduled for 3.1.3 but is not yet fixed even :)

Also, does deploy suffer from this? Or it is not affected?

[jira-importer]

Tamas Cservenak commented

Well, your "howerer..." paragraph is true not only for install, but for the whole project. As this comment suggests (https://github.com/mojohaus/versions/issues/1071#issuecomment-2109926461) the plugin is simply put "too late", your model is already built. Hence, uninterpolated $revision is there, everywhere.

Simply put your contribution to versions maven plugin, while it gets the needed info, and even puts it into (already built) project properties, is not enough, as model would need to be fully rebuilt to take new property into account. Also, it goes against what is explained on this page as well: https://maven.apache.org/maven-ci-friendly.html The property MUST BE available ahead of time, is clearly explained there as well. When a plugin executes, it is already too late, as model was built, phases calculated, and then goal is invoked (but as I said, model was already built, so you'd somehow need to "trigger model rebuild", which is not possible in Maven).

[jira-importer]

Tamas Cservenak commented

Isn't something like this (just throwing ideas, not test) what you really want? And no need for plugin or anythig:

$REVISION=`git rev-parse HEAD` && mvn install -Drevision=$REVISION

[jira-importer]

Jimisola Laursen commented

This was quite a downer sadly as dynamic versioning would simplify a lot of things for us in our CI/CD.

No, there is quite some logic behind getting the version (source code here) since it's based on tags (semantic versioned such, i.e. x.y.z) and number of commits (for when the current/last commit is not tagged, then number of commits to that tag is counted).

Python has this available for build systems poetry(poetry-dynamic-versioning and hatch (hatch-vcs).

Having dynamic versioning with Maven, i.e. not manually changing {$project.version } but using scm/vcs tags is our preferred choice (minimizing e.g. the risk of creating a release without updating the project.version).

And "trigger model rebuild" is not coming with Maven 4? fingers crossed

So, the only option is to build an extension then? Looked at https://maven.apache.org/guides/mini/guide-using-extensions.html#core-extension before. Would a build extension suffice or does it have to be core extension? Registration of a build extension seems easier to manage. But, if I were to change the goal to an extension, are you then able to configure when you want this and only this extension to be active?

[jira-importer]

Tamas Cservenak commented

Try this: https://github.com/maveniverse/nisse Note: jgit-source is trivial, not (yet) what you want, but PRs are welcome!

[jira-importer]

Jimisola Laursen commented

Wow. Thanks. I'll do. Currently, on a deadline with test tool at work but I'll look at it next week for sure and see I can migrate the functionality from Versions Maven plugin. Never written an extension before, so I'll have to read up on how one can configure the extension (from pom.xml's configuration as well as from command line).

[jira-importer]

Jimisola Laursen commented

We always want to be able use Maven in the same manner locally as in our CI/CD. Sometimes we use profiles (e.g. time consuming goals) that aren't activated locally but one can always opt-in. I'm suspecting that there is a challenge in how the one can configure the loading of an extension as per https://maven.apache.org/guides/mini/guide-using-extensions.html#core-extension:

1. Registered via extension jar in {}${maven.home}/lib/ext{}: clutters Maven installations and/or potentially requires rebuild of container with pre-installed Maven (seems tedious to make it work with Maven Wrapper)
2. Registered via CLI argument mvn -Dmaven.ext.class.path=extension.jar: not a good option for something permanent
3. Registered via .mvn/extensions.xml file:per project only

It seems as if none of these options are a good fit for our use-case (unless I misunderstood them):

• should work with both maven and maven wrapper
• should work globally (i.e. not on project level)
• should work both locally and in ci/cd, some extensions not be active per default but available for opt-in

I noticed that there are a few (old) issues here that would meet our use-case, namely settings.xml and profiles:

• https://issues.apache.org/jira/browse/MNG-6903
• https://issues.apache.org/jira/browse/MNG-5820
• https://issues.apache.org/jira/browse/MNG-6961

 

 

[jira-importer]

Tamas Cservenak commented

Jimisola Laursen Well, I do hope you talk about your own projects ("in shop" projects), as then the .mvn/extensions.xml solution could be very easily implemented, no? That would work with both mvn and mvnw, unsure what "should work globally" means, it would work for whoever builds it (human on workstation or CI)...

[jira-importer]

Jimisola Laursen commented

I think I found a solution for us using MAVEN_ARGS, but I'll have to test it so that MAVEN_ARGS can be used to point out a core extension with

MAVEN_ARGS="-Dmaven.ext.class.path=extension.jar"

Yes, inhouse projects and we have a lot of them.

We avoid configuration files in projects like the plague because it does not scale (easy to forget to add and/or update). 

If we have to make a change then we have to do it in all those projects and it's configuration for lombok, maven, log4j, sonarqube and so on. Every single tool has the mindset that it's just "a configuration file more". It's sadly not. It needs to be maintained and version controlled. It was, e.g. only in May this year that renovate got support for .mvn/extensions.xml.

If it is something that should be applied to "all" our projects then we prefer to configure that globally and handle the outliers separately.

Globally in this case is that the configuration can be set in a way that it applies to all projects being run/build on that machine for a user and ideally it can also be overridden on project level or similar.

That is, it does not matter if it's an actual user on a machine or in CI/CD. The only difference in this case is that Registered via CLI argument mvn -Dmaven.ext.class.path=extension.jar would be very simple to add and maintain in CI/CD ({}and this is when I recalled seeing something about MAVEN_ARGS for Maven 3.9+{}).

E.g. for mypy we have a global configuration file in ~/.config/mypy/mypy.ini that can be overridden on project level if needed.

I also had an idea of :

 

<profiles>
    <profile>
        <id>default-options</id>
        <properties>
            <maven.ext.class.path>extension.jar</maven.ext.class.path>
        </properties>
    </profile>
</profiles> 

but, I'm not sure that that is the same as providing it on the command line as -D<...> or if settings.xml is read too late when it comes to core extensions. Been looking for documentation on it and searching for execution diagrams without luck.

[jira-importer]

Jimisola Laursen commented

I tried 

mvn dependency:copy -Dartifact=eu.maveniverse.maven.nisse:extension3:0.1.1 -DoutputDirectory=/tmp/maven-core-extensions-test/extension3-0.1.1.jar

$MAVEN_ARGS="-Dmaven.ext.class.path=/tmp/maven-core-extensions-test/extension3-0.1.1.jar" mvn eu.maveniverse.maven.plugins:toolbox:dump -Dverbose -N

I don't get any nisse user properties (my maven.ext.class.path is set to extension3 per above) but do get a "java.lang.RuntimeException: runtime".

Might have stumbled on an issue with transitive dependencies since there is only a jar file and not a Maven GAV. https://stackoverflow.com/questions/69114665/defining-maven-extensions-globally-as-a-gav

[jira-importer]

Tamas Cservenak commented

It does not work like it, ext classpath does not resolve, it needs full transitive hull. Example https://gist.github.com/cstamas/d4850aa724c46940a41295e9f124863b

(if not obvious: copy-paste gav-classpath output to -Dmaven.ext.class.path)

[jira-importer]

Jimisola Laursen commented

Ok. Thanks for the info. So, being able to specify extensions using GAV in settings.xml would be really use full then.

Might be able to use maven-shade och maven-assembly to create one bfj (big fat jar).