iceberg icon indicating copy to clipboard operation
iceberg copied to clipboard

Cannot find constructor for interface org.apache.parquet.column.page.PageWriteStore?

Open williamliu111 opened this issue 11 months ago • 2 comments

Query engine

No response

Question

use iceberg java api to write data, The error message is as follows:

Exception in thread "main" java.lang.ExceptionInInitializerError at org.apache.iceberg.parquet.Parquet$WriteBuilder.build(Parquet.java:268) at Iceberg.writeFile(Iceberg.java:113) at Iceberg.main(Iceberg.java:200) Caused by: java.lang.RuntimeException: Cannot find constructor for interface org.apache.parquet.column.page.PageWriteStore Missing org.apache.parquet.hadoop.ColumnChunkPageWriteStore(org.apache.parquet.hadoop.CodecFactory$BytesCompressor,org.apache.parquet.schema.MessageType,org.apache.parquet.bytes.ByteBufferAllocator,int) [java.lang.NoSuchMethodException: org.apache.parquet.hadoop.ColumnChunkPageWriteStore.(org.apache.parquet.hadoop.CodecFactory$BytesCompressor, org.apache.parquet.schema.MessageType, org.apache.parquet.bytes.ByteBufferAllocator, int)] at org.apache.iceberg.common.DynConstructors$Builder.build(DynConstructors.java:236) at org.apache.iceberg.parquet.ParquetWriter.(ParquetWriter.java:58) ... 3 more

my part pom.xml: <groupId>org.apache.parquet</groupId> <artifactId>parquet-column</artifactId> 1.12.0 <groupId>org.apache.parquet</groupId> <artifactId>parquet-jackson</artifactId> 1.12.0 <groupId>org.apache.parquet</groupId> <artifactId>parquet-hadoop</artifactId> 1.12.0 <groupId>org.apache.parquet</groupId> <artifactId>parquet-avro</artifactId> 1.11.1 <groupId>org.apache.parquet</groupId> <artifactId>parquet-common</artifactId> 1.12.0 <groupId>org.apache.parquet</groupId> <artifactId>parquet-format-structures</artifactId> 1.12.0 <groupId>org.apache.hive</groupId> <artifactId>hive-storage-api</artifactId> 2.7.3 <groupId>org.apache.avro</groupId> <artifactId>avro</artifactId> 1.9.0 <groupId>org.apache.spark</groupId> <artifactId>spark-sql_2.12</artifactId> 3.3.0

williamliu111 avatar Feb 25 '24 13:02 williamliu111

It looks you have two versions of parquet in your dependencies, 1.11.1 and 1.12.0

manuzhang avatar Feb 27 '24 06:02 manuzhang