DataX icon indicating copy to clipboard operation
DataX copied to clipboard

Published 20 hours ago • verified publisher icon alibaba

如何修改hdfs的reader hdfswriter 让他们支持hadoop3?

Open xuliancheng563 opened this issue 1 year ago • 2 comments

xuliancheng563 avatar Jan 11 '24 04:01 xuliancheng563

将 <hive.version>3.1.0</hive.version> <hadoop.version>3.1.4</hadoop.version>版本修改成 报错 程序包parquet.schema不存在

xuliancheng563 avatar Jan 11 '24 04:01 xuliancheng563

找到了 解决方案，在hdfswriter 当中加上 依赖 ：

 <dependency>
            <groupId>com.twitter</groupId>
            <artifactId>parquet-hadoop-bundle</artifactId>
            <version>1.6.0rc3</version>
        </dependency>

xiaomaisuii avatar Feb 29 '24 08:02 xiaomaisuii