incubator-streampark icon indicating copy to clipboard operation
incubator-streampark copied to clipboard

[Bug] maven conflict

Open learner1212 opened this issue 1 year ago • 1 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

Java Version

1.8

Scala Version

2.12.x

StreamPark Version

2.1.0

Flink Version

1.16.1

deploy mode

yarn-application

What happened

when paste maven dependencies, throw NoSuchMethodError.

in streampark-flink-packer module, maven-aether-provider.3.3.9.jar include aether-api.1.0.2.v20150114.jar but we want 1.1.0 version

Error Exception

02:31:37.362 [streampark-build-pipeline-executor-1] ERROR org.apache.streampark.flink.packer.pipeline.impl.FlinkYarnApplicationBuildPipeline - [StreamPark] [streampark-packer] building pipeline has failed. | appName=load_uv
java.util.concurrent.ExecutionException: java.lang.NoSuchMethodError: org.apache.maven.model.ReportSet.isInherited()Z
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:206)
	at org.apache.streampark.flink.packer.pipeline.BuildPipeline.$anonfun$launch$1(BuildPipeline.scala:136)
	at scala.util.Try$.apply(Try.scala:213)
	at org.apache.streampark.flink.packer.pipeline.BuildPipeline.launch(BuildPipeline.scala:129)
	at org.apache.streampark.flink.packer.pipeline.BuildPipeline.launch$(BuildPipeline.scala:127)
	at org.apache.streampark.flink.packer.pipeline.impl.FlinkYarnApplicationBuildPipeline.launch(FlinkYarnApplicationBuildPipeline.scala:32)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.NoSuchMethodError: org.apache.maven.model.ReportSet.isInherited()Z
	at org.apache.maven.model.merge.MavenModelMerger.mergeReportPlugin_ReportSets(MavenModelMerger.java:603)
	at org.apache.maven.model.merge.ModelMerger.mergeReportPlugin(ModelMerger.java:1328)
	at org.apache.maven.model.inheritance.DefaultInheritanceAssembler$InheritanceModelMerger.mergeReporting_Plugins(DefaultInheritanceAssembler.java:295)
	at org.apache.maven.model.merge.ModelMerger.mergeReporting(ModelMerger.java:1260)
	at org.apache.maven.model.merge.ModelMerger.mergeModelBase_Reporting(ModelMerger.java:653)
	at org.apache.maven.model.merge.ModelMerger.mergeModelBase(ModelMerger.java:522)
	at org.apache.maven.model.merge.ModelMerger.mergeModel(ModelMerger.java:116)
	at org.apache.maven.model.merge.MavenModelMerger.mergeModel(MavenModelMerger.java:75)
	at org.apache.maven.model.merge.ModelMerger.merge(ModelMerger.java:111)
	at org.apache.maven.model.inheritance.DefaultInheritanceAssembler.assembleModelInheritance(DefaultInheritanceAssembler.java:58)
	at org.apache.maven.model.building.DefaultModelBuilder.assembleInheritance(DefaultModelBuilder.java:721)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:377)
	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:320)
	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:198)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:287)
	at org.apache.streampark.flink.packer.maven.MavenTool$.$anonfun$resolveArtifacts$4(MavenTool.scala:189)
	at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:237)
	at scala.collection.immutable.Set$Set4.foreach(Set.scala:208)
	at scala.collection.TraversableLike.map(TraversableLike.scala:237)
	at scala.collection.TraversableLike.map$(TraversableLike.scala:230)
	at scala.collection.AbstractSet.scala$collection$SetLike$$super$map(Set.scala:51)
	at scala.collection.SetLike.map(SetLike.scala:104)
	at scala.collection.SetLike.map$(SetLike.scala:104)
	at scala.collection.AbstractSet.map(Set.scala:51)
	at org.apache.streampark.flink.packer.maven.MavenTool$.resolveArtifacts(MavenTool.scala:189)
	at org.apache.streampark.flink.packer.pipeline.impl.FlinkYarnApplicationBuildPipeline.$anonfun$buildProcess$4(FlinkYarnApplicationBuildPipeline.scala:60)
	at org.apache.streampark.flink.packer.pipeline.BuildPipeline.$anonfun$execStep$1(BuildPipeline.scala:102)
	at scala.util.Try$.apply(Try.scala:213)
	at org.apache.streampark.flink.packer.pipeline.BuildPipeline.execStep(BuildPipeline.scala:97)
	at org.apache.streampark.flink.packer.pipeline.BuildPipeline.execStep$(BuildPipeline.scala:96)
	at org.apache.streampark.flink.packer.pipeline.impl.FlinkYarnApplicationBuildPipeline.execStep(FlinkYarnApplicationBuildPipeline.scala:32)
	at org.apache.streampark.flink.packer.pipeline.impl.FlinkYarnApplicationBuildPipeline.buildProcess(FlinkYarnApplicationBuildPipeline.scala:58)
	at org.apache.streampark.flink.packer.pipeline.impl.FlinkYarnApplicationBuildPipeline.buildProcess(FlinkYarnApplicationBuildPipeline.scala:32)
	at org.apache.streampark.flink.packer.pipeline.BuildPipeline$$anon$1.call(BuildPipeline.scala:134)
	at org.apache.streampark.flink.packer.pipeline.BuildPipeline$$anon$1.call(BuildPipeline.scala:133)
	... 4 common frames omitted

Screenshots

No response

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!(您是否要贡献这个PR?)

Code of Conduct

learner1212 avatar May 12 '23 04:05 learner1212

I don't think streampark can resovle maven dependency in depth.
When I have <!-- https://mvnrepository.com/artifact/com.ververica/flink-connector-mysql-cdc --> <dependency> <groupId>com.ververica</groupId> <artifactId>flink-connector-mysql-cdc</artifactId> <version>2.3.0</version> </dependency>

MavenTool.scala will only resolve 1st depth deps, which are flink-connector-debezium, debezium-connector-mysql . Stop digging further deps.

shenyi0828 avatar Apr 26 '24 03:04 shenyi0828