flinkStreamSQL icon indicating copy to clipboard operation
flinkStreamSQL copied to clipboard

通过Yarn模式提交任务时报错Caused by: java.lang.ClassNotFoundException: com.dtstack.flink.sql.source.kafka.KafkaConsumer010

Open lijianfeng6666526 opened this issue 5 years ago • 3 comments

### sql文件内容如下: CREATE TABLE sourceKafkaTable( id int as id, name varchar as name )WITH( type ='kafka10', bootstrapServers ='spark01:9092,spark02:9092', zookeeperQuorum ='spark01:2181,spark02:2181/brokers', offsetReset ='latest', topic ='flink1', parallelism ='1' );

CREATE TABLE sideMysqlTable( id int, cname varchar, PRIMARY KEY(id), PERIOD FOR SYSTEM_TIME )WITH( type='mysql', url='jdbc:mysql://192.168.181.15:3306/hive?charset=utf8', userName='root', password='6666526', tableName='side_order', cache ='LRU', cacheSize ='10000', cacheTTLMs ='60000', parallelism ='1', partitionedJoin='false' );

CREATE TABLE sinkMysqlTable( id int, name varchar, cname varchar )WITH( type ='mysql', url ='jdbc:mysql://192.168.181.15:3306/hive?charset=utf8', userName ='root', password ='6666526', tableName ='result_order', parallelism ='1' );

insert into sinkMysqlTable select d.id, d.name, d.cname from (select a.id,a.name,b.cname from sourceKafkaTable a join sideMysqlTable b on a.id=b.id) as d

================================================================ ### 提交的命令如下: sh submit.sh -sql /home/flinksql.txt -name flinkstream -remoteSqlPluginPath /home/FlinkStreamSQL-1.8.0-new/plugins -localSqlPluginPath /home/FlinkStreamSQL-1.8.0-new/plugins -mode yarn -yarnconf /home/hadoop-2.6.0/etc/hadoop -flinkconf /home/flink-1.8.0/conf -confProp {"time.characteristic":"EventTime","sql.checkpoint.interval":10000} -yarnSessionConf {"yid":"application_1582170401251_0003"}

================================================================ ### 报如下错误: 图片

lijianfeng6666526 avatar Feb 23 '20 01:02 lijianfeng6666526

@lijianfeng6666526 你用yarnPer 和 -pluginLoadMode shipfile 模式试试,yarnsession模式我们的测试的比较少。

todd5167 avatar Feb 24 '20 01:02 todd5167

用yarnPer会保其他错误

lijianfeng6666526 avatar Feb 24 '20 07:02 lijianfeng6666526

遇到同样的问题,yarnPer无法正常运行报错和上面是一样的。 -pluginLoadMode shipfile

hengyuan avatar Mar 05 '20 05:03 hengyuan