alibabacloud-jindodata icon indicating copy to clipboard operation
alibabacloud-jindodata copied to clipboard

hdfs_to_hdfs 加入参数--preserveMeta报错

Open zhenghang2 opened this issue 1 year ago • 1 comments

Error: java.io.IOException: Exception raised while copying meta org.apache.hadoop.hdfs.DistributedFileSystem cannot be cast to com.aliyun.jindodata.common.JindoHadoopSystem at com.aliyun.jindodata.distcp.task.CpTask.handleSetMetaFailures(CpTask.java:292) at com.aliyun.jindodata.distcp.task.CpTask.syncDirMetaStatus(CpTask.java:335) at com.aliyun.jindodata.distcp.task.CpTask.loop(CpTask.java:124) at com.aliyun.jindodata.distcp.task.CpTask.loop(CpTask.java:41) at com.aliyun.jindodata.distjob.task.DistTask.run(DistTask.java:62) at com.aliyun.jindodata.distcp.DistCpMapper.map(DistCpMapper.java:41) at com.aliyun.jindodata.distcp.DistCpMapper.map(DistCpMapper.java:17) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:178) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:172) Caused by: java.lang.ClassCastException: org.apache.hadoop.hdfs.DistributedFileSystem cannot be cast to com.aliyun.jindodata.common.JindoHadoopSystem at com.aliyun.jindodata.distcp.task.CpTask.syncDirMetaStatus(CpTask.java:330) ... 13 more 这个是什么原因?大佬们解答一下?

zhenghang2 avatar Jun 02 '23 08:06 zhenghang2

Error: java.io.IOException: Exception raised while copying meta org.apache.hadoop.hdfs.DistributedFileSystem cannot be cast to com.aliyun.jindodata.common.JindoHadoopSystem at com.aliyun.jindodata.distcp.task.CpTask.handleSetMetaFailures(CpTask.java:292) at com.aliyun.jindodata.distcp.task.CpTask.syncDirMetaStatus(CpTask.java:335) at com.aliyun.jindodata.distcp.task.CpTask.loop(CpTask.java:124) at com.aliyun.jindodata.distcp.task.CpTask.loop(CpTask.java:41) at com.aliyun.jindodata.distjob.task.DistTask.run(DistTask.java:62) at com.aliyun.jindodata.distcp.DistCpMapper.map(DistCpMapper.java:41) at com.aliyun.jindodata.distcp.DistCpMapper.map(DistCpMapper.java:17) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:178) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:172) Caused by: java.lang.ClassCastException: org.apache.hadoop.hdfs.DistributedFileSystem cannot be cast to com.aliyun.jindodata.common.JindoHadoopSystem at com.aliyun.jindodata.distcp.task.CpTask.syncDirMetaStatus(CpTask.java:330) ... 13 more 这个是什么原因?大佬们解答一下?

麻烦提供一下运行的环境,如HDFS版本,及使用的jindo distcp版本。并且描述一下需求,如果只是单纯的HDFS to HDFS场景,其实可以考虑社区版本的hadoop distcp。

yanbinyang avatar Jun 03 '23 14:06 yanbinyang