maven-shade-plugin icon indicating copy to clipboard operation
maven-shade-plugin copied to clipboard

Published 20 hours ago • verified publisher icon apache

[MSHADE-384] maven building hang when using multithread on jdk 11

Open jira-importer opened this issue 4 years ago • 4 comments

Zhengqi Zhang opened MSHADE-384 and commented

When I use mvn clean package and add -T to the command, the console log stops on a certain line. then I use -X to start the debug log, and I find that I have been looping through something, as follows.When I get rid of the -T parameter, everything is fine.When I switched to JDK8, it was fine to use multithreaded packaging

!image-2021-02-24-20-04-58-353.png|width=943,height=574!

I get the thread stack, as shown in the figure below.

!image-2021-02-24-20-06-47-076.png|width=838,height=388!

 

Affects: 3.2.1, 3.2.4

Attachments:

Issue Links:

  • FLINK-21499 maven package hang when using multithread on jdk 11

0 votes, 5 watchers

jira-importer avatar Feb 24 '21 12:02 jira-importer

Michael Osipov commented

Can you retry with Maven from master?

jira-importer avatar Feb 24 '21 13:02 jira-importer

Zhengqi Zhang commented

I compiled the latest code from the master branch of the maven-shade-plugin, i.e. 3.3.0-snapshot, installed it to the local repository, and then introduced it in my project. But this time a new problem appeared. The log is as follows:

 

[ERROR] Failed to execute goal org.commonjava.maven.plugins:directory-maven-plugin:0.4-SNAPSHOT:highest-basedir (directories) on project flink-table: Cannot find a single highest directory for this project set. First two candidates directories don't share a common root. -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.commonjava.maven.plugins:directory-maven-plugin:0.4-SNAPSHOT:highest-basedir (directories) on project flink-table: Cannot find a single highest directory for this project set. First two candidates directories don't share a common root.

 

After searching, I found that mavenSession in 3.3.0-snapshot incorrectly identified my top-level directory. My project is a multi-module project. In 3.2.1, the top-level project was identified as MavenProject: org.apache.flink:flink-parent:1.12-SNAPSHOT @ /Users/zhangzhengqi5/IdeaProjects/flink/pom.xml, that's true, but in 3.3.0 - the SNAPSHOT, the top-level project is identified as MavenProject: org.apache.flink:flink-parent:1.12-SNAPSHOT @ /Users/zhangzhengqi5/IdeaProjects/flink/target/dependency-reduced-pom.xml. I think the 3.3.0-snapshot version has a problem with initializing mavenSession.

 

I think this is a different problem from the multithreaded compilation failure mentioned above。

jira-importer avatar Feb 25 '21 04:02 jira-importer

Andrey Turbanov commented

Reproduce with Amazon Corretto java 8 too. -T 4 was specified.

openjdk version "1.8.0_312"
OpenJDK Runtime Environment Corretto-8.312.07.1 (build 1.8.0_312-b07)
OpenJDK 64-Bit Server VM Corretto-8.312.07.1 (build 25.312-b07, mixed mode)

Stack traces:


"BuilderThread 3" #15 prio=5 os_prio=0 tid=0x00007f7044364000 nid=0x3dd7c2 runnable [0x00007f702a0ef000]
   java.lang.Thread.State: RUNNABLE
	at org.jdom2.Element.isAncestor(Element.java:1052)
	at org.jdom2.ContentList.checkPreConditions(ContentList.java:222)
	at org.jdom2.ContentList.add(ContentList.java:244)
	at org.jdom2.Element.addContent(Element.java:950)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.insertAtPreferredLocation(MavenJDOMWriter.java:296)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.iterateExclusion(MavenJDOMWriter.java:492)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.updateDependency(MavenJDOMWriter.java:1339)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.iterateDependency(MavenJDOMWriter.java:390)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.updateModel(MavenJDOMWriter.java:1628)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.write(MavenJDOMWriter.java:2164)
	at org.apache.maven.plugins.shade.pom.PomWriter.write(PomWriter.java:75)
	at org.apache.maven.plugins.shade.mojo.ShadeMojo.rewriteDependencyReducedPomIfWeHaveReduction(ShadeMojo.java:1169)
	at org.apache.maven.plugins.shade.mojo.ShadeMojo.createDependencyReducedPom(ShadeMojo.java:1098)
	at org.apache.maven.plugins.shade.mojo.ShadeMojo.execute(ShadeMojo.java:599)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:200)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:196)

"BuilderThread 2" #14 prio=5 os_prio=0 tid=0x00007f7044298800 nid=0x3dd7c1 runnable [0x00007f702a1f0000]
   java.lang.Thread.State: RUNNABLE
	at org.jdom2.Element.isAncestor(Element.java:1052)
	at org.jdom2.ContentList.checkPreConditions(ContentList.java:222)
	at org.jdom2.ContentList.add(ContentList.java:244)
	at org.jdom2.Element.addContent(Element.java:950)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.insertAtPreferredLocation(MavenJDOMWriter.java:296)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.iterateExclusion(MavenJDOMWriter.java:492)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.updateDependency(MavenJDOMWriter.java:1339)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.iterateDependency(MavenJDOMWriter.java:390)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.updateModel(MavenJDOMWriter.java:1628)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.write(MavenJDOMWriter.java:2164)
	at org.apache.maven.plugins.shade.pom.PomWriter.write(PomWriter.java:75)
	at org.apache.maven.plugins.shade.mojo.ShadeMojo.rewriteDependencyReducedPomIfWeHaveReduction(ShadeMojo.java:1169)
	at org.apache.maven.plugins.shade.mojo.ShadeMojo.createDependencyReducedPom(ShadeMojo.java:1098)
	at org.apache.maven.plugins.shade.mojo.ShadeMojo.execute(ShadeMojo.java:599)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:200)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:196)

"BuilderThread 1" #13 prio=5 os_prio=0 tid=0x00007f7044298000 nid=0x3dd7c0 runnable [0x00007f702a2f1000]
   java.lang.Thread.State: RUNNABLE
	at org.jdom2.Element.isAncestor(Element.java:1052)
	at org.jdom2.ContentList.checkPreConditions(ContentList.java:222)
	at org.jdom2.ContentList.add(ContentList.java:244)
	at org.jdom2.Element.addContent(Element.java:950)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.insertAtPreferredLocation(MavenJDOMWriter.java:296)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.iterateExclusion(MavenJDOMWriter.java:492)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.updateDependency(MavenJDOMWriter.java:1339)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.iterateDependency(MavenJDOMWriter.java:390)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.updateModel(MavenJDOMWriter.java:1628)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.write(MavenJDOMWriter.java:2164)
	at org.apache.maven.plugins.shade.pom.PomWriter.write(PomWriter.java:75)
	at org.apache.maven.plugins.shade.mojo.ShadeMojo.rewriteDependencyReducedPomIfWeHaveReduction(ShadeMojo.java:1169)
	at org.apache.maven.plugins.shade.mojo.ShadeMojo.createDependencyReducedPom(ShadeMojo.java:1098)
	at org.apache.maven.plugins.shade.mojo.ShadeMojo.execute(ShadeMojo.java:599)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:200)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:196)

"BuilderThread 0" #12 prio=5 os_prio=0 tid=0x00007f7044362800 nid=0x3dd7ba runnable [0x00007f702ad16000]
   java.lang.Thread.State: RUNNABLE
	at org.jdom2.Element.isAncestor(Element.java:1052)
	at org.jdom2.ContentList.checkPreConditions(ContentList.java:222)
	at org.jdom2.ContentList.add(ContentList.java:244)
	at org.jdom2.Element.addContent(Element.java:950)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.insertAtPreferredLocation(MavenJDOMWriter.java:296)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.iterateExclusion(MavenJDOMWriter.java:492)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.updateDependency(MavenJDOMWriter.java:1339)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.iterateDependency(MavenJDOMWriter.java:390)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.updateModel(MavenJDOMWriter.java:1628)
	at org.apache.maven.plugins.shade.pom.MavenJDOMWriter.write(MavenJDOMWriter.java:2164)
	at org.apache.maven.plugins.shade.pom.PomWriter.write(PomWriter.java:75)
	at org.apache.maven.plugins.shade.mojo.ShadeMojo.rewriteDependencyReducedPomIfWeHaveReduction(ShadeMojo.java:1169)
	at org.apache.maven.plugins.shade.mojo.ShadeMojo.createDependencyReducedPom(ShadeMojo.java:1098)
	at org.apache.maven.plugins.shade.mojo.ShadeMojo.execute(ShadeMojo.java:599)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:200)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:196)

Build hanged for long time. After removing -T 4 got build success.

jira-importer avatar Jan 14 '22 15:01 jira-importer

Robert Stupp commented

Maybe MSHADE-413?

jira-importer avatar Mar 22 '22 08:03 jira-importer