go-offline-maven-plugin icon indicating copy to clipboard operation
go-offline-maven-plugin copied to clipboard

Published 20 hours ago • verified publisher icon qaware

plugins not fetched

Open avodonosov opened this issue 4 years ago • 11 comments

Plugins seem to be not fetched at all.

I've tried on two projects: https://github.com/redisson/redisson and https://github.com/eclipse/winery. After successfully completing the resolve-dependencies, their build in offline mode fails.

For redisson: [ERROR] Plugin org.apache.maven.plugins:maven-surefire-plugin:2.12.4 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.apache.maven.plugins:maven-surefire-plugin:jar:2.12.4 has not been downloaded from it before. -> [Help 1]

For winery: [ERROR] Plugin org.apache.maven.plugins:maven-jar-plugin:3.0.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.apache.maven.plugins:maven-jar-plugin:jar:3.0.0 has not been downloaded from it before. -> [Help 1]

These plugins are not dynamic dependencies, they are directly mentioned in the build -> plugins sections of the corresponding projects.

avodonosov avatar Nov 29 '20 19:11 avodonosov

Thanks for the report. I was able to reproduce the issue. I will look into at as soon as I have time.

Theoderich avatar Nov 30 '20 08:11 Theoderich

I have the same issue, with a different plugin. I reference it in the pom under build/plugins and I even tried adding it as a dynamic plugin as well. Neither worked.

Plugin org.apache.maven.plugins:maven-clean-plugin:3.1.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.apache.maven.plugins:maven-clean-plugin:jar:3.1.0 has not been downloaded from it before.

<properties>
    <maven-clean-plugin.version>3.1.0</maven-clean-plugin.version>
    <go-offline-maven-plugin.version>1.2.5</go-offline-maven-plugin.version>
</properties>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-clean-plugin</artifactId>
            <version>${maven-clean-plugin.version}</version>
        </plugin>

        <plugin>
            <groupId>de.qaware.maven</groupId>
            <artifactId>go-offline-maven-plugin</artifactId>
            <version>${go-offline-maven-plugin.version}</version>
            <configuration>
                <dynamicDependencies>
                    <dynamicDependency>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-clean-plugin</artifactId>
                        <version>${maven-clean-plugin.version}</version>
                        <repositoryType>PLUGIN</repositoryType>
                    </dynamicDependency>   
                </dynamicDependencies>
            </configuration>
        </plugin>
    </plugins>
</build>

mvn de.qaware.maven:go-offline-maven-plugin:resolve-dependencies -Dmaven.repo.local=./binaries

For some reason it downloaded the version 2.5 into org\apache\maven\plugins\maven-clean-plugin and I cannot figure out where this version could come from. There is no dependency with that version anywhere in my repository. Maybe because I am using properties to resolve the version?

Edit: 2.5 is the default version of the maven-clean-plugin and is coming from a "naked" module without any parent pom, without a go-offline definition. Once I define the version of the maven-clean-plugin other plugins start to fail (downloading the wrong version) - so I think the issue comes from pom's using the default configuration somehow.

My actual usecase is a quite complex multi-module setup including parent pom's defining the go-offline-maven-plugin and all other plugins. So each of the sub-modules is "going offline" one by one, all into the same repository. The child poms are very sparse compared to the shared parent pom.

ScrambledRK avatar Dec 02 '20 16:12 ScrambledRK

As a workaround I "went offline" by deleting my local repository, building my project and saving the resulting local repository. Even then I encountered errors when trying to build offline using said repository via -Dmaven.repo.local=./binaries.

The issue was that empty aggregator poms, not defining anything beside child modules, still required some plugins that were not downloaded. Ensuring that even aggregator poms define all the same plugins resolved this issue for me.

I have yet to try the de.qaware.maven:go-offline-maven-plugin:resolve-dependencies once again, but I am short on time at the moment.

ScrambledRK avatar Dec 03 '20 14:12 ScrambledRK

I have identified the issue: The plugin filters duplicate Plugins before fetching. Unfortunately, if a plugin was defined multiple times with different versions, all but the first version was filtered. Different <Dependencies> sections of the same plugin where also ignored. In have completely removed the filtering since the performance-impact seems more or less non-existant. The fix is pushed to master. You can try it by building the master and explicitly running the 1.2.8-SNAPSHOT version of the plugin. I will release it to maven-central soon if no further bugs are reported

Theoderich avatar Dec 07 '20 09:12 Theoderich

After building and running the current SNAPSHOT version for winery with mvn -Pjava de.qaware.maven:go-offline-maven-plugin:1.2.8-SNAPSHOT:resolve-dependencies I noticed the following in the output:

[INFO] --- go-offline-maven-plugin:1.2.8-SNAPSHOT:resolve-dependencies (default-cli) @ winery --- [WARNING] The POM for com.github.ust-edmm.edmm:edmm-core:jar:v1.0.6 is missing, no dependency information available [ERROR] Error downloading dependencies for project [ERROR] Failure to find com.github.ust-edmm.edmm:edmm-core:jar:v1.0.6 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced [ERROR] Error downloading plugin dependencies for project [ERROR] Failure to find org.ethereum:solcJ-all:jar:0.4.25 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced [WARNING] Failure to find com.github.ust-edmm.edmm:edmm-core:jar:v1.0.6 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced [WARNING] Failure to find org.ethereum:solcJ-all:jar:0.4.25 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced ... [INFO] BUILD SUCCESS

Don't know is that bad or not. Investigating in the dependency tree built with mvn org.apache.maven.plugins:maven-dependency-plugin:3.1.2:tree > target/dependency-tree I don't see ethereum:solcJ-al at all.

avodonosov avatar Dec 07 '20 19:12 avodonosov

When executing winery build after that with mvn -X -Pjava --offline package I've got:

[ERROR] Failed to execute goal org.web3j:web3j-maven-plugin:4.2.0:generate-sources (generate-sources) on project org.eclipse.winery.accountability: Execution generate-sources of goal org.web3j:web3j-maven-plugin:4.2.0:generate-sources failed: Plugin org.web3j:web3j-maven-plugin:4.2.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.ethereum:solcJ-all:jar:0.4.25 has not been downloaded from it before. -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.web3j:web3j-maven-plugin:4.2.0:generate-sources (generate-sources) on project org.eclipse.winery.accountability: Execution generate-sources of goal org.web3j:web3j-maven-plugin:4.2.0:generate-sources failed: Plugin org.web3j:web3j-maven-plugin:4.2.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.ethereum:solcJ-all:jar:0.4.25 has not been downloaded from it before.

	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution generate-sources of goal org.web3j:web3j-maven-plugin:4.2.0:generate-sources failed: Plugin org.web3j:web3j-maven-plugin:4.2.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.ethereum:solcJ-all:jar:0.4.25 has not been downloaded from it before.
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
	... 20 more
Caused by: org.apache.maven.plugin.PluginResolutionException: Plugin org.web3j:web3j-maven-plugin:4.2.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.ethereum:solcJ-all:jar:0.4.25 has not been downloaded from it before.
	at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolveInternal(DefaultPluginDependenciesResolver.java:218)
	at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolve(DefaultPluginDependenciesResolver.java:149)
	at org.apache.maven.plugin.internal.DefaultMavenPluginManager.createPluginRealm(DefaultMavenPluginManager.java:400)
	at org.apache.maven.plugin.internal.DefaultMavenPluginManager.setupPluginRealm(DefaultMavenPluginManager.java:372)
	at org.apache.maven.plugin.DefaultBuildPluginManager.getPluginRealm(DefaultBuildPluginManager.java:231)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:102)
	... 21 more
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.ethereum:solcJ-all:jar:0.4.25 has not been downloaded from it before.
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444)
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:367)
	at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolveInternal(DefaultPluginDependenciesResolver.java:210)
	... 26 more
Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.ethereum:solcJ-all:jar:0.4.25 has not been downloaded from it before.
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:391)
	... 29 more
Caused by: org.eclipse.aether.transfer.RepositoryOfflineException: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode
	at org.eclipse.aether.internal.impl.DefaultOfflineController.checkOffline(DefaultOfflineController.java:72)
	at org.eclipse.aether.internal.impl.Utils.checkOffline(Utils.java:115)
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:387)
	... 29 more
[ERROR] 
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :org.eclipse.winery.accountability

avodonosov avatar Dec 07 '20 19:12 avodonosov

When repeating the experiment with fresh maven repo specified through -Dmaven.repo.local=/some/dir//winery-m2-repo, the resolve-dependencies succeeds with the same ERROR and WARNING, but the offline build fails earlier:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test (default-test) on project org.eclipse.winery.model.tosca: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test failed: Plugin org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.codehaus.plexus:plexus-utils:jar:1.1 has not been downloaded from it before. -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test (default-test) on project org.eclipse.winery.model.tosca: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test failed: Plugin org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.codehaus.plexus:plexus-utils:jar:1.1 has not been downloaded from it before.

	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test failed: Plugin org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.codehaus.plexus:plexus-utils:jar:1.1 has not been downloaded from it before.
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
	... 20 more
Caused by: org.apache.maven.plugin.PluginResolutionException: Plugin org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.codehaus.plexus:plexus-utils:jar:1.1 has not been downloaded from it before.
	at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolveInternal(DefaultPluginDependenciesResolver.java:218)
	at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolve(DefaultPluginDependenciesResolver.java:149)
	at org.apache.maven.plugin.internal.DefaultMavenPluginManager.createPluginRealm(DefaultMavenPluginManager.java:400)
	at org.apache.maven.plugin.internal.DefaultMavenPluginManager.setupPluginRealm(DefaultMavenPluginManager.java:372)
	at org.apache.maven.plugin.DefaultBuildPluginManager.getPluginRealm(DefaultBuildPluginManager.java:231)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:102)
	... 21 more
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.codehaus.plexus:plexus-utils:jar:1.1 has not been downloaded from it before.
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444)
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:367)
	at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolveInternal(DefaultPluginDependenciesResolver.java:210)
	... 26 more
Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.codehaus.plexus:plexus-utils:jar:1.1 has not been downloaded from it before.
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:391)
	... 29 more
Caused by: org.eclipse.aether.transfer.RepositoryOfflineException: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode
	at org.eclipse.aether.internal.impl.DefaultOfflineController.checkOffline(DefaultOfflineController.java:72)
	at org.eclipse.aether.internal.impl.Utils.checkOffline(Utils.java:115)
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:387)
	... 29 more
[ERROR] 
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :org.eclipse.winery.model.tosca

avodonosov avatar Dec 07 '20 20:12 avodonosov

@Theoderich , thank you for addressing the issue, some problem seems to remain, sorry.

How do you think, why it is so difficult to fetch exactly those dependencies as used during real build? Is it some unfortunate maven architecture?

Ideally the go-offline would execute exactly the same procedure normal maven build uses, instead of replicating the logic. It would require maven to preform some kind of dry-run - execute everything as in normal build but without actually running build operations. In some sense similary apt-get install --dry-run in Debian.

avodonosov avatar Dec 07 '20 20:12 avodonosov

Unfortunately I know of no way to dry-run the maven build. I started the go-offline-maven-plugin because the official maven-dependency-plugin:go-offline goal was not even able to handle reactor builds at the time and my pull request fixing the issue was not merged after over a year. Ideally, the official plugin would be able to go-offline correctly. If there was an easy way to achive this, I am sure the official plugin would use it.

In the end there will never be a "perfect" solution to this problem unless the maven core team addresses it. As it is now, plugins can just dynamicaly load other plugins in their code without ever declaring it in their pom. I see no way to solve that with a plugin

Theoderich avatar Dec 08 '20 09:12 Theoderich

Yes, I faced several problems with the official dependency plugin go-offline functionality, and when wanted to report it to the plugin jira found many similar tickets already reported, one of them suggesting to use the go-offline plugin instead.

avodonosov avatar Dec 08 '20 16:12 avodonosov

I have identified the issue: The plugin filters duplicate Plugins before fetching. Unfortunately, if a plugin was defined multiple times with different versions, all but the first version was filtered. Different sections of the same plugin where also ignored. In have completely removed the filtering since the performance-impact seems more or less non-existant. The fix is pushed to master. You can try it by building the master and explicitly running the 1.2.8-SNAPSHOT version of the plugin. I will release it to maven-central soon if no further bugs are reported

It seems this issue is not limited to Plugins, but to all dependencies - I have a multimodule project with dependencyManagement defined in root POM and one of the modules overrides some dependency versions.

After running mvn de.qaware.maven:go-offline-maven-plugin:1.2.8:resolve-dependencies my local Maven cache has only one version of the JARs which had overridden versions, instead of two (general and the override) required to build the entire project in offline mode.

MiguelWeezardo avatar Jan 25 '23 11:01 MiguelWeezardo