[Feature-11488][Datax]Datax can submit to YARN
https://github.com/apache/dolphinscheduler/issues/11488
Instructions:
Configure the environment variables DATAX_ON_YARN_DEPEND_JAR and DATAX_HDFS_PATH to run DATAX ON YARN
Parameter explanation: DATAX_ON_YARN_DEPEND_JAR: https://github.com/duhanmin/datax-on-yarn Compile and package
DATAX_HDFS_PATH: You need to upload the datax installation package to yarn
example:
#export DATAX_ON_YARN_DEPEND_JAR=${DATAX_ON_YARN_DEPEND_JAR:-/opt/soft/datax/datax-on-yarn-1.0.0.jar}
#DATAX_HDFS_HOME is HDFS path
#export DATAX_HDFS_PATH=${DATAX_HDFS_HOME:-/hdfs/opt/hadoop/datax.tar.gz}
使用方法:
配置环境变量DATAX_ON_YARN_DEPEND_JAR和DATAX_HDFS_PATH就可以运行DATAX ON YARN
参数解释: DATAX_ON_YARN_DEPEND_JAR: https://github.com/duhanmin/datax-on-yarn 编译打包
DATAX_HDFS_PATH: 需要将datax安装包上传到yarn
例子:
#export DATAX_ON_YARN_DEPEND_JAR=${DATAX_ON_YARN_DEPEND_JAR:-/opt/soft/datax/datax-on-yarn-1.0.0.jar}
#DATAX_HDFS_HOME is HDFS path
#export DATAX_HDFS_PATH=${DATAX_HDFS_HOME:-/hdfs/opt/hadoop/datax.tar.gz}
Hi @duhanmin , please associate the issue and correct the title.
@duhanmin Hello, thanks for submitting this PR and it's a good feature. I've added some comments and please take a look.