Parallel maven build warnings

[alexandrvb]

Description

I've got following warnings when trying to use docker-maven-plugin in parallel mode (mvn docker:build -T):

[WARNING] *****************************************************************
[WARNING] * Your build is requesting parallel execution, but project      *
[WARNING] * contains the following plugin(s) that have goals not marked   *
[WARNING] * as @threadSafe to support parallel building.                  *
[WARNING] * While this /may/ work fine, please look for plugin updates    *
[WARNING] * and/or request plugins be made thread-safe.                   *
[WARNING] * If reporting an issue, report it against the plugin in        *
[WARNING] * question, not against maven-core                              *
[WARNING] *****************************************************************
[WARNING] The following plugins are not marked @threadSafe in project:
[WARNING] io.fabric8:docker-maven-plugin:0.23.0
[WARNING] Enable debug to see more precisely which goals are not marked @threadSafe.
[WARNING] *****************************************************************

Info

• d-m-p version : 0.23.0
• Maven version (mvn -v) : 3.5.0

[wborn]

Even though there is a warning we've been using it to build containers without any problems in parallel builds for months now. :slightly_smiling_face:

[yufei-cai]

I'm building docker images in parallel from multiple modules independent of each other. Docker-maven-plugin 0.23.0 hangs at the 'build' goal in maven 3.5.2, but it runs through without problems in maven 3.3.9. Does anyone want to work on this and need a reproducer?

[rhuss]

@yufei-cai it would be awesome if you could send in a stack trace when it hangs (CTRL-\ or 'kill -QUIT' to the Maven process).

Alternatively, if you can send in a sample script that would be very helpful, too.

Thanks!

[ffendt]

@rhuss : on behalf of @yufei-cai i'm sending in a threaddump.log.

[rhuss]

@ffendt Thanks ! I'm currently in christmas break but will have a look asap.

[burtsevyg]

maven 3.5.4 dmp 0.27.2 openjdk 8 191 mvn -T 1C -B -Dmaven.test.failure.ignore clean deploy

build and push goal doesn't executed

            <plugin>
                <groupId>io.fabric8</groupId>
                <artifactId>docker-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <id>docker-build-and-push-${project.artifactId}</id>
                        <phase>verify</phase>
                        <goals>
                            <goal>build</goal>
                            <goal>push</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

[burtsevyg]

@rohanKanojia Could you comment on #905 please. Is it possible to resolve it?

[mhermosi]

Versions and execution command:

Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T14:33:14-04:00)
Maven home: /Users/user/Development/maven
Java version: 12.0.1, vendor: Oracle Corporation, 
runtime: /Library/Java/JavaVirtualMachines/openjdk-12.0.1.jdk/Contents/Home

command:
mvn -B -s ./settings.xml -Dmaven.test.skip=true clean install docker:build docker:push -P private-registry -T 1C

I have another thing about parallel build. When using -T option on maven and on single or multiple executions the plugin lose the defined properties:

• properties:

${container.registry}, ${image.prefix} and ${project.version}

• pom.xml extract:

<plugin>
   <groupId>io.fabric8</groupId>
      <artifactId>docker-maven-plugin</artifactId>
      <executions>
            <execution>
                  <id>demo-ui-build</id>
                  <configuration>
                        <registry>${container.registry}</registry>
                        <images>
                           <image>
                              <name>${image.prefix}/${project.artifactId}:${project.version}</name>
                              <build>
                                 <dockerFileDir>${project.basedir}</dockerFileDir>
                                 <dockerFile>Dockerfile</dockerFile>
                              </build>
                           </image>
                     </images>
                  </configuration>
            </execution>
            <execution>
                  <id>app-ui-build</id>
                  <configuration>
                        <registry>${container.registry}</registry>
                        <images>
                           <image>
                              <name>${image.prefix}/app-ui:${project.version}</name>
                              <build>
                                 <dockerFileDir>${project.basedir}</dockerFileDir>
                                 <dockerFile>Dockerfile</dockerFile>
                              </build>
                           </image>
                     </images>
                  </configuration>
            </execution>
         </executions>
  </plugin>

For some reason ${project.artifactId} is not lost.... but ${project.version} is lost and seems to be defaulted to latest

The weird thing is that ${image.prefix} seems to be getting the last part of the parent pom.xml groupId I mean.. if artifact is something like com.domain.xxx.yyy:artifactName:0.0.1-SNAPSHOT and ${image.prefix} = base then the result of image name is yyy/artifactName:latest instead of base/artifactName:0.0.1-SNAPSHOT

${container.registry} is being set on the profile definition.

In conclusion, I think this plugin should review its coding to support thread-safety so we can use the whole potential of

[mhermosi]

Found the issue.

I was executing maven-exec-plugin on package phase and also docker plugin on package phase. I moved maven-exec-plugin to compile (because I am doing some javascript build) and docker to install and seems to be working fine ... even for parallel build

[mattnelson]

@rhuss Here is a stack during a parallel build when it appears to be hung, locking on object 0x0000000782adfa28

Culprit in maven https://github.com/apache/maven/blob/maven-3.6.1/maven-model-builder/src/main/java/org/apache/maven/model/interpolation/StringSearchModelInterpolator.java#L296

Based on the change log for 3.6.2 I believe this issue may be addressed. https://issues.apache.org/jira/issues/?jql=project%20%3D%20MNG%20AND%20fixVersion%20%3D%203.6.2

thread with the lock

"BuilderThread 1" #27 prio=5 os_prio=31 cpu=15490.35ms elapsed=90.94s tid=0x00007febea75c800 nid=0x730f runnable  [0x000070000f068000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.Class.isAssignableFrom([email protected]/Native Method)
	at jdk.internal.reflect.UnsafeFieldAccessorImpl.ensureObj([email protected]/UnsafeFieldAccessorImpl.java:57)
	at jdk.internal.reflect.UnsafeObjectFieldAccessorImpl.get([email protected]/UnsafeObjectFieldAccessorImpl.java:36)
	at java.lang.reflect.Field.get([email protected]/Field.java:418)
	at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction$MapField.doInterpolate(StringSearchModelInterpolator.java:430)
	at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction$CacheField.interpolate(StringSearchModelInterpolator.java:302)
	- locked <0x0000000782adfa28> (a java.lang.reflect.Field)
	at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction$CacheItem.interpolate(StringSearchModelInterpolator.java:275)
	at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction.traverseObjectWithParents(StringSearchModelInterpolator.java:160)
	at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction.run(StringSearchModelInterpolator.java:134)
	at java.security.AccessController.doPrivileged([email protected]/Native Method)
	at org.apache.maven.model.interpolation.StringSearchModelInterpolator.interpolateObject(StringSearchModelInterpolator.java:81)
	at org.apache.maven.model.interpolation.StringSearchModelInterpolator.interpolateModel(StringSearchModelInterpolator.java:64)
	at org.apache.maven.model.building.DefaultModelBuilder.interpolateModel(DefaultModelBuilder.java:781)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:388)
	at org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:171)
	at org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:340)
	at org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:297)
	at org.apache.maven.project.artifact.MavenMetadataSource.retrieveRelocatedProject(MavenMetadataSource.java:577)
	at org.apache.maven.project.artifact.MavenMetadataSource.retrieve(MavenMetadataSource.java:190)
	at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:551)
	at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
	at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
	at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
	at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
	at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
	at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.collect(DefaultLegacyArtifactCollector.java:148)
	at org.apache.maven.artifact.resolver.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:504)
	at org.apache.maven.repository.legacy.LegacyRepositorySystem.resolve(LegacyRepositorySystem.java:367)
	at org.apache.maven.plugins.assembly.artifact.DefaultDependencyResolver.resolveTransitively(DefaultDependencyResolver.java:233)
	at org.apache.maven.plugins.assembly.artifact.DefaultDependencyResolver.resolve(DefaultDependencyResolver.java:119)
	at org.apache.maven.plugins.assembly.artifact.DefaultDependencyResolver.resolveDependencySets(DefaultDependencyResolver.java:155)
	at org.apache.maven.plugins.assembly.archive.phase.DependencySetAssemblyPhase.execute(DependencySetAssemblyPhase.java:95)
	at org.apache.maven.plugins.assembly.archive.DefaultAssemblyArchiver.createArchive(DefaultAssemblyArchiver.java:181)
	at io.fabric8.maven.docker.assembly.DockerAssemblyManager.createAssemblyArchive(DockerAssemblyManager.java:466)
	at io.fabric8.maven.docker.assembly.DockerAssemblyManager.createDockerTarArchive(DockerAssemblyManager.java:115)
	at io.fabric8.maven.docker.service.ArchiveService.createArchive(ArchiveService.java:123)
	at io.fabric8.maven.docker.service.ArchiveService.createArchive(ArchiveService.java:118)
	at io.fabric8.maven.docker.service.BuildService.buildImage(BuildService.java:132)
	at io.fabric8.maven.docker.service.BuildService.buildImage(BuildService.java:70)
	at io.fabric8.maven.docker.BuildMojo.buildAndTag(BuildMojo.java:74)
	at io.fabric8.maven.docker.BuildMojo.processImageConfig(BuildMojo.java:105)
	at io.fabric8.maven.docker.BuildMojo.executeInternal(BuildMojo.java:61)
	at io.fabric8.maven.docker.AbstractDockerMojo.execute(AbstractDockerMojo.java:230)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:202)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:198)
	at java.util.concurrent.FutureTask.run([email protected]/FutureTask.java:264)
	at java.util.concurrent.Executors$RunnableAdapter.call([email protected]/Executors.java:515)
	at java.util.concurrent.FutureTask.run([email protected]/FutureTask.java:264)
	at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1128)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run([email protected]/Thread.java:834)

thread waiting on the lock

"BuilderThread 23" #283 prio=5 os_prio=31 cpu=4023.80ms elapsed=83.59s tid=0x00007febf194d800 nid=0xc91f waiting for monitor entry  [0x000070000f779000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction$CacheField.interpolate(StringSearchModelInterpolator.java:296)
	- waiting to lock <0x0000000782adfa28> (a java.lang.reflect.Field)
	at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction$CacheItem.interpolate(StringSearchModelInterpolator.java:275)
	at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction.traverseObjectWithParents(StringSearchModelInterpolator.java:160)
	at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction.run(StringSearchModelInterpolator.java:134)
	at java.security.AccessController.doPrivileged([email protected]/Native Method)
	at org.apache.maven.model.interpolation.StringSearchModelInterpolator.interpolateObject(StringSearchModelInterpolator.java:81)
	at org.apache.maven.model.interpolation.StringSearchModelInterpolator.interpolateModel(StringSearchModelInterpolator.java:64)
	at org.apache.maven.model.building.DefaultModelBuilder.interpolateModel(DefaultModelBuilder.java:781)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:394)
	at org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:171)
	at org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:340)
	at org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:297)
	at org.apache.maven.project.artifact.MavenMetadataSource.retrieveRelocatedProject(MavenMetadataSource.java:577)
	at org.apache.maven.project.artifact.MavenMetadataSource.retrieve(MavenMetadataSource.java:190)
	at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:551)
	at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
	at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
	at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
	at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
	at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
	at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.collect(DefaultLegacyArtifactCollector.java:148)
	at org.apache.maven.artifact.resolver.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:504)
	at org.apache.maven.repository.legacy.LegacyRepositorySystem.resolve(LegacyRepositorySystem.java:367)
	at org.apache.maven.plugins.assembly.artifact.DefaultDependencyResolver.resolveTransitively(DefaultDependencyResolver.java:233)
	at org.apache.maven.plugins.assembly.artifact.DefaultDependencyResolver.resolve(DefaultDependencyResolver.java:119)
	at org.apache.maven.plugins.assembly.artifact.DefaultDependencyResolver.resolveDependencySets(DefaultDependencyResolver.java:155)
	at org.apache.maven.plugins.assembly.archive.phase.DependencySetAssemblyPhase.execute(DependencySetAssemblyPhase.java:95)
	at org.apache.maven.plugins.assembly.archive.DefaultAssemblyArchiver.createArchive(DefaultAssemblyArchiver.java:181)
	at io.fabric8.maven.docker.assembly.DockerAssemblyManager.createAssemblyArchive(DockerAssemblyManager.java:466)
	at io.fabric8.maven.docker.assembly.DockerAssemblyManager.createDockerTarArchive(DockerAssemblyManager.java:115)
	at io.fabric8.maven.docker.service.ArchiveService.createArchive(ArchiveService.java:123)
	at io.fabric8.maven.docker.service.ArchiveService.createArchive(ArchiveService.java:118)
	at io.fabric8.maven.docker.service.BuildService.buildImage(BuildService.java:132)
	at io.fabric8.maven.docker.service.BuildService.buildImage(BuildService.java:70)
	at io.fabric8.maven.docker.BuildMojo.buildAndTag(BuildMojo.java:74)
	at io.fabric8.maven.docker.BuildMojo.processImageConfig(BuildMojo.java:105)
	at io.fabric8.maven.docker.BuildMojo.executeInternal(BuildMojo.java:61)
	at io.fabric8.maven.docker.AbstractDockerMojo.execute(AbstractDockerMojo.java:230)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:202)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:198)
	at java.util.concurrent.FutureTask.run([email protected]/FutureTask.java:264)
	at java.util.concurrent.Executors$RunnableAdapter.call([email protected]/Executors.java:515)
	at java.util.concurrent.FutureTask.run([email protected]/FutureTask.java:264)
	at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1128)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run([email protected]/Thread.java:834)

[cowwoc]

Greetings from almost-2025 (!!) What's the status of this issue? Are you able to mark this plugin thread-safe?

[cowwoc]

@mattnelson Does this problem still occur with the latest version of Maven (now 3.9.9)?

[mattnelson]

Does this problem still occur with the latest version of Maven (now 3.9.9)?

It has been a long time(years) since I last saw this error, so it was likely fixed in 3.6.x. Looking at the change logs I see a few things in 3.6.2 and 3.6.3 that could be the fix. Either way, maven users should be using 3.9.x anyway now.

[eliasbalasis]

I believe this relates to #1327

[eliasbalasis]

Does this problem still occur with the latest version of Maven (now 3.9.9)?

It has been a long time(years) since I last saw this error, so it was likely fixed in 3.6.x. Looking at the change logs I see a few things in 3.6.2 and 3.6.3 that could be the fix. Either way, maven users should be using 3.9.x anyway now.

I intend to give this a try soon.

[eliasbalasis]

Unfortunately, the error is still present with io.fabric8:docker-maven-plugin:0.44.0 , "Maven 3.9.9" and io.takari.maven:takari-smart-builder:1.0.2

However, using some Maven trickery, it is possible to force the Maven modules making use of io.fabric8:docker-maven-plugin to run sequentially during a Maven parallel build.

[jalphonso]

Also seeing this issue on Maven 3.9.9 Is there an estimated timeframe to resolve?