chunjun icon indicating copy to clipboard operation
chunjun copied to clipboard

[Bug] 数据同步remoteChunJunDistDir 参数不生效

Open zhourui999 opened this issue 2 years ago • 0 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

What happened

一: 参数设置如下 argsList.add("-remoteChunJunDistDir"); argsList.add("hdfs://nameHAservice/user/badmin/ruitest/usrlib/chunjun-dist");

image

即使我填写是hdfs地址,但是这段远程代码只能判断本地的jar是否存在,而不能判断远程的jar是否存在。导致后续一系列的问题

二:后面我通过设置yarn.provided.usrlib.dir 参数来解决这个无法配置远程chunjun包的问题 stringBuilder .append("{") .append(""execution.target":"yarn-per-job"") .append(", ") .append(""yarn.provided.usrlib.dir":"/user/badmin/ruitest/usrlib"") .append(", ") .append(""yarn.provided.lib.dirs":"/user/badmin/ruitest/flink-1.16.1/lib"") .append(", ") .append(""yarn.application.queue":"root.root"") .append("} ");

image 但是上面的这段代码依然会导致将版本的chunjun connector包上传到hdfs,最后我将该段代码注释了。解决了这个问题。

希望remoteChunJunDistDir参数能够真实起作用

What you expected to happen

remoteChunJunDistDir设置后,可以使用hdfs上的chunjun jar

How to reproduce

参数设置如上

Anything else

No response

Version

master

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

zhourui999 avatar Jun 20 '23 10:06 zhourui999