DataX icon indicating copy to clipboard operation
DataX copied to clipboard

hdfsreader: fastjson error in the latest version

Open yixingwang0517 opened this issue 2 years ago • 3 comments

2023-10-09 17:48:09.632 [main] INFO Engine - { "content":[ { "reader":{ "name":"hdfsreader", "parameter":{ "column":[ "" ], "defaultFS":"hdfs://10.130.1.6:9000", "encoding":"UTF-8", "fieldDelimiter":"\t", "fileType":"text", "path":"/user/datax/student.txt" } }, "writer":{ "name":"mysqlwriter", "parameter":{ "column":[ "id", "name" ], "connection":[ { "jdbcUrl":"jdbc:mysql://10.130.1.6:3306/datax?useSSL=false", "table":[ "student2" ] } ], "password":"*****", "username":"root", "writeMode":"insert" } } } ], "setting":{ "speed":{ "channel":"1" } } }

2023-10-09 17:48:09.654 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false 2023-10-09 17:48:09.655 [main] INFO JobContainer - DataX jobContainer starts job. 2023-10-09 17:48:09.656 [main] INFO JobContainer - Set jobId = 0 2023-10-09 17:48:09.669 [job-0] INFO HdfsReader$Job - init() begin... 2023-10-09 17:48:10.117 [job-0] ERROR JobContainer - Exception when job run java.lang.StackOverflowError: null at java.base/java.lang.ClassLoader.definePackage(ClassLoader.java:2106) ~[na:na] at java.base/java.lang.ClassLoader.lambda$packages$4(ClassLoader.java:2372) ~[na:na] at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[na:na] at java.base/java.util.concurrent.ConcurrentHashMap$ValueSpliterator.forEachRemaining(ConcurrentHashMap.java:3603) ~[na:na] at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[na:na] at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[na:na] at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:550) ~[na:na] at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260) ~[na:na] at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:517) ~[na:na] at java.base/java.lang.ClassLoader.getDefinedPackages(ClassLoader.java:2271) ~[na:na] at com.alibaba.fastjson2.writer.FieldWriterObjectFuncFinal.getFieldValue(FieldWriterObjectFuncFinal.java:26) ~[fastjson2-2.0.23.jar:na] at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:52) ~[fastjson2-2.0.23.jar:na] at com.alibaba.fastjson2.writer.ObjectWriter5.write(ObjectWriter5.java:86) ~[fastjson2-2.0.23.jar:na] at com.alibaba.fastjson2.writer.OWG_4_8_Module.write(Unknown Source) ~[na:na] at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:81) ~[fastjson2-2.0.23.jar:na] at com.alibaba.fastjson2.writer.ObjectWriter5.write(ObjectWriter5.java:90) ~[fastjson2-2.0.23.jar:na] at com.alibaba.fastjson2.writer.OWG_4_8_Module.write(Unknown Source) ~[na:na] at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:81) ~[fastjson2-2.0.23.jar:na] at com.alibaba.fastjson2.writer.ObjectWriter5.write(ObjectWriter5.java:90) ~[fastjson2-2.0.23.jar:na] at com.alibaba.fastjson2.writer.OWG_4_8_Module.write(Unknown Source) ~[na:na] at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:81) ~[fastjson2-2.0.23.jar:na] at com.alibaba.fastjson2.writer.ObjectWriter5.write(ObjectWriter5.java:90) ~[fastjson2-2.0.23.jar:na] at com.alibaba.fastjson2.writer.OWG_4_8_Module.write(Unknown Source) ~[na:na] at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:81) ~[fastjson2-2.0.23.jar:na] at com.alibaba.fastjson2.writer.ObjectWriter5.write(ObjectWriter5.java:90) ~[fastjson2-2.0.23.jar:na] at com.alibaba.fastjson2.writer.OWG_4_8_Module.write(Unknown Source) ~[na:na] at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:81) ~[fastjson2-2.0.23.jar:na] at com.alibaba.fastjson2.writer.ObjectWriter5.write(ObjectWriter5.java:90) ~[fastjson2-2.0.23.jar:na] at com.alibaba.fastjson2.writer.OWG_4_8_Module.write(Unknown Source) ~[na:na] at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:81) ~[fastjson2-2.0.23.jar:na] at com.alibaba.fastjson2.writer.ObjectWriter5.write(ObjectWriter5.java:90) ~[fastjson2-2.0.23.jar:na] at com.alibaba.fastjson2.writer.OWG_4_8_Module.write(Unknown Source) ~[na:na] at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:81) ~[fastjson2-2.0.23.jar:na] at com.alibaba.fastjson2.writer.ObjectWriter5.write(ObjectWriter5.java:90) ~[fastjson2-2.0.23.jar:na] at com.alibaba.fastjson2.writer.OWG_4_8_Module.write(Unknown Source) ~[na:na] at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:81) ~[fastjson2-2.0.23.jar:na]

yixingwang0517 avatar Oct 09 '23 09:10 yixingwang0517

这个错误是不是跟数据有关系?

FuYouJ avatar Oct 09 '23 13:10 FuYouJ

DFSUtil初始化conf时,注释掉日志输出LOG.info(String.format("hadoopConfig details:%s", this.hadoopConf.toString()));

MichaelLi505 avatar Jan 31 '24 09:01 MichaelLi505

遇到了相同的问题,请问下问题解决了么?

wangxinjie2 avatar Apr 03 '24 05:04 wangxinjie2