maven icon indicating copy to clipboard operation
maven copied to clipboard

Feature Request: CLI Option to Force Download of Both POM and JAR Files for All Dependencies

Open kerimovkhikmet opened this issue 3 months ago • 10 comments

New feature, improvement proposal

Currently, Maven downloads POM files for all dependencies, but only downloads JAR files when they are required for the build. This behavior causes issues when local Maven repositories are used by other build tools, such as sbt (via Ivy or Coursier), which often fail when a POM exists locally, but the corresponding JAR does not.

Proposed Feature: Add a new CLI option, for example:

mvn dependency:resolve -DforceDownloadAllArtifacts=true

or

mvn -DdownloadAllArtifacts

When enabled, Maven should download both the POM and JAR files for every direct and transitive dependency defined in the project.

Use Case: This feature would make local Maven repositories fully compatible with sbt and similar tools that expect both POM and JAR artifacts to exist locally. It would also help in offline builds and multi-tool environments where artifact completeness is critical.

Expected Behavior: When the new flag is set, Maven should:

  • Resolve all dependencies (direct and transitive).
  • Download both POM and JAR files for each dependency.
  • Store them in the local repository, regardless of whether the build currently needs the JAR.

Motivation: This feature would prevent inconsistent local repository states that currently lead to missing JAR resolution failures in sbt’s Ivy or Coursier-based dependency resolution. It would also provide a more predictable and complete local cache for cross-build-tool interoperability.

As noted in the Coursier's documentation:

Avoid using ~/.m2/repository as a repository if you can. This is maven's internal file-system cache and it's not meant to be read by external tools. By using it you are risking running into some issues like e.g. directories with POMs inside but without corresponding jar files. You can read more about it here and here.

This highlights a real and recurring issue caused by Maven’s current behavior. Adding an option to force JAR downloads would eliminate such inconsistencies and improve interoperability across tools.

kerimovkhikmet avatar Oct 09 '25 16:10 kerimovkhikmet

This would be an issue for resolver or m-dependency-p, not Maven.

But, as your pointed doco explains this is not how Maven works. It is nicely explained here:

The reason for that is that maven will first download only poms to create a dependency tree. Then versions will be chosen based on maven's version conflict resolution strategy and only then maven will download jars for the picked versions.

For example, when version range is used, that range may contain tens or even more "candidates" (who's POM will be downloaded), but not all the JARs will ever be needed.

And even more, Maven "enhanced" (used by Maven 3) local repository tracks origins of artifacts, and existence of that metadata is a must for correct mode of work (to let Maven figure out things for you), so "sharing" is not so simple. Finally, Maven transparently can use "split local repository" as well, that completely changes the layout on disk...

cstamas avatar Oct 09 '25 17:10 cstamas

Btw, you can achieve similar thing using Toolbox:

mvn toolbox:gav-resolve-transitive -Dgav=org.slf4j:slf4j-api:1.7.36,org.slf4j:slf4j-simple:1.7.36

or even using JBang

jbang toolbox@maveniverse resolve-transitive org.slf4j:slf4j-api:1.7.36,org.slf4j:slf4j-simple:1.7.36

or if you have a Maven project (from within project):

mvn toolbox:resolve-transitive -DfailOnLogicalFailure=false

cstamas avatar Oct 09 '25 17:10 cstamas

I haven't used it for a long time (I think it was when I had a 56k modem...) but should that not be what you want?

https://maven.apache.org/plugins/maven-dependency-plugin/go-offline-mojo.html

laeubi avatar Oct 10 '25 07:10 laeubi

Btw, you can achieve similar thing using Toolbox:

mvn toolbox:gav-resolve-transitive -Dgav=org.slf4j:slf4j-api:1.7.36,org.slf4j:slf4j-simple:1.7.36

or even using JBang

jbang toolbox@maveniverse resolve-transitive org.slf4j:slf4j-api:1.7.36,org.slf4j:slf4j-simple:1.7.36

or if you have a Maven project (from within project):

mvn toolbox:resolve-transitive -DfailOnLogicalFailure=false

So, I have a subcomponent built via Maven, and then it is used to build the main project via sbt. I have tried to run the suggested command in my Maven project as

mvn eu.maveniverse.maven.plugins:toolbox:0.13.5:resolve-transitive -DfailOnLogicalFailure=false

and it didn't result in a local repository containing all JARs. As a result, my main project's build via sbt still fails when encountering dependencies that have only POMs locally.

kerimovkhikmet avatar Oct 10 '25 09:10 kerimovkhikmet

I haven't used it for a long time (I think it was when I had a 56k modem...) but should that not be what you want?

https://maven.apache.org/plugins/maven-dependency-plugin/go-offline-mojo.html

As was stated in the previous comment, I have a subcomponent built via Maven, and then it is used to build the main project via sbt. I have tried to run the suggested tool in my Maven project via

mvn org.apache.maven.plugins:maven-dependency-plugin:3.9.0:go-offline

and it also didn't result in a local repository containing all JARs. And then subsequently, my main project's build via sbt failed once again when encountering dependencies that have only POMs locally.

kerimovkhikmet avatar Oct 10 '25 09:10 kerimovkhikmet

Maybe it would be better to make sbt smarter? I must confess I don't really understand the workflow here and how the dependencies are actually managed downloading "everything" would certainly work but will increase the sieze enormously. And if maven do not download it why sbt is needing it?

In any case what I do in such case is using adding an execution of dependency:copy-dependencies to the parent pom bound e.g. to compile phase. This will then for sure download everything and even put it into a folder that you can direct your tool to consume it from (or just ignore it if you like).

laeubi avatar Oct 10 '25 09:10 laeubi

This would be an issue for resolver or m-dependency-p, not Maven.

But, as your pointed doco explains this is not how Maven works. It is nicely explained here:

The reason for that is that maven will first download only poms to create a dependency tree. Then versions will be chosen based on maven's version conflict resolution strategy and only then maven will download jars for the picked versions.

For example, when version range is used, that range may contain tens or even more "candidates" (who's POM will be downloaded), but not all the JARs will ever be needed.

Alright, I get what you mean, and see possible complications.

And even more, Maven "enhanced" (used by Maven 3) local repository tracks origins of artifacts, and existence of that metadata is a must for correct mode of work (to let Maven figure out things for you), so "sharing" is not so simple.

Do you mean that the addition of an option to remove all non-used POMs somewhere during the end phases, and respective local folders, also could not be applicable?

Finally, Maven transparently can use "split local repository" as well, that completely changes the layout on disk...

Well, then, can you please share the link to the documentation with examples and details on how it works?

kerimovkhikmet avatar Oct 10 '25 10:10 kerimovkhikmet

Maybe it would be better to make sbt smarter? I must confess I don't really understand the workflow here and how the dependencies are actually managed downloading "everything" would certainly work but will increase the sieze enormously. And if maven do not download it why sbt is needing it?

Not sbt per se, but the tools used for managing artifacts for sbt: ivy and coursier. During artifact resolution, each of those tools finds a POM in the local ~/.m2 repository, and assumes that it is a complete repository, i.e., has every required artifact, when in reality it does not. Now, one way is to make both ivy and coursier "smarter", and other resolver tools in the ecosystem, is to not make that assumption based on the availability of the single POM file in the local repository. Another way is to make something with Maven's resolver, to somehow control the creation of the incomplete local repositories.

In any case what I do in such case is using adding an execution of dependency:copy-dependencies to the parent pom bound e.g. to compile phase. This will then for sure download everything and even put it into a folder that you can direct your tool to consume it from (or just ignore it if you like).

Thank you, I'll give it a try and will let you know the results.

kerimovkhikmet avatar Oct 10 '25 14:10 kerimovkhikmet

Well, then, can you please share the link to the documentation with examples and details on how it works?

Some doco links:

  • https://maven.apache.org/resolver-archives/resolver-1.9.24/local-repository.html
  • https://maveniverse.eu/blog/2025/03/17/never-say-never/#local-repositories
  • and finally, you have local repo chained local repo

cstamas avatar Oct 13 '25 15:10 cstamas

Also, one very important bit, that people tend to forget:

I as a publisher may have a simple single module project w/ packaging=jar and and I publish it to Central under GAV.

You, as a consumer are not obliged to consume my JAR, as you can still depend on my project with GAV + type=pom, which is totally "legal" and valid, and in this case, your Maven will pull my POM only and never JAR. Use case for this may be simply "reusing dependencies" without reusing published code (weird, but still valid).

Moral of the story: Maven publishing and Maven consuming are intentionally disconnected in this way (people keep confusing packaging, type and extension, assuming they are same; which is not true). Also, "consumer" Maven (run by you while consuming my project) completely ignores packaging of my project (that is packaging=jar), and listens to your instructions, that is type=pom.

Typical example is plain JAR, that can be produced by packaging jar, but also with things like takari-jar for example. Both output will produce JAR that can be consumed as type=jar (the default, usually not written), and there is no need for you to assume "how it was built" (was it jar or takari-jar or my-super-duper-jar-packaging at build time).

tl;dr: artifacts in your local repository that are cached (so not locally installed) have packaging fully ignored by Maven. The POM packaging has "meaning" ONLY during build-time and are ignored at consumption-time.

cstamas avatar Oct 14 '25 10:10 cstamas