读取ftp数据的时候会报错

[shaolei7788]

读取ftp数据的时候，FtpSourceFactory#createSource，ftpConfig.setColumn 每个FieldConf的index为null，会导致FtpInputFormat#nextRecordInternal 里的分支永远不会执行，

需要修改一下 将图1的图片改为 ftpConfig.setColumn(syncConf.getReader().getFieldList());

[zoudaokoulife]

@shaolei7788 任务的json 发下。

[shaolei7788]

json 内容如下：

{ "job":{ "content":[ { "reader":{ "parameter":{ "path":"/sftpuser/emp.txt", "protocol":"sftp", "port":22, "isFirstLineHeader":false, "host":"ip", "privateKeyPath": "/root/.ssh/id_rsa", "column": [ { "name": "comm", "type": "DOUBLE" }, { "name": "deptno", "type": "INT" }, { "name": "empno", "type": "INT" }, { "name": "ename", "type": "VARCHAR" }, { "name": "hiredate", "type": "DATE" } ], "password": "123456", "fieldDelimiter":",", "encoding":"utf-8", "username":"sftpuser" }, "name":"ftpreader" }, "writer": { "name": "mysqlwriter", "parameter": { "column": [ { "name": "comm", "type": "DOUBLE" }, { "name": "deptno", "type": "INT" }, { "name": "empno", "type": "INT" }, { "name": "ename", "type": "VARCHAR" }, { "name": "hiredate", "type": "DATE" } ], "connection": [ { "jdbcUrl": "jdbc:mysql://ip:3306/test?useSSL=false", "table": [ "emp2" ], "schema": "" } ], "password": "pa88w0rd", "username": "root" } } } ], "setting":{ "restore":{ "maxRowNumForCheckpoint":0, "isRestore":false, "restoreColumnName":"", "restoreColumnIndex":0 }, "errorLimit":{ "record":100 }, "speed":{ "bytes":0, "channel":1 } } } }

[shaolei7788]

emp.txt内容如下

,20,7369,SMITH,1980-12-17 300,30,7499,ALLEN,1981-02-20 500,30,7521,WARD,1981-02-22 ,20,7566,JONES,1981-04-02 1400,30,7654,MARTIN,1981-09-28 ,30,7698,BLAKE,1981-05-01 ,10,7782,CLARK,1981-06-09 ,20,7788,SCOTT,1987-07-13 ,10,7839,KING,1981-11-17 0,30,7844,TURNER,1981-09-08 ,20,7876,ADAMS,1987-07-13 ,30,7900,JAMES,1981-12-03 ,20,7902,FORD,1981-12-03 ,10,7934,MILLER,1982-01-23

[lujisen]

@shaolei7788 ftp_ftp 一直报错空指针，有没有遇到过，我直接拿的官方的ftp_ftp.json文件修改的， { "job":{ "content":[ { "reader":{ "parameter":{ "path":"/var/ftp/test/data.csv", "protocol":"ftp", "port":21, "isFirstLineHeader":false, "host":"xx.xx.xx.xx", "column":[ { "index":0, "type":"int", "name": "id" }, { "index":1, "type":"int", "name": "status" }, { "index":2, "type":"string", "name": "des" } ], "password":"123456***", "fieldDelimiter":",", "encoding":"utf-8", "username":"ftptest" }, "name":"ftpreader" }, "writer":{ "parameter": { "path": "/var/ftp/test/data.csv", "protocol": "ftp", "port": 21, "writeMode": "overwrite", "host": "xx.xx.xx.xx", "column":[ { "index":0, "type":"int", "name": "id" }, { "index":1, "type":"int", "name": "status" }, { "index":2, "type":"string", "name": "des" } ], "password": "123456***", "fieldDelimiter": ",", "encoding": "utf-8", "username": "ftptest" }, "name":"ftpwriter" } } ], "setting":{ "restore":{ "maxRowNumForCheckpoint":0, "isRestore":false, "restoreColumnName":"", "restoreColumnIndex":0 }, "errorLimit":{ "record":100 }, "speed":{ "bytes":0, "channel":1 } } } }