kafka-connect-storage-common icon indicating copy to clipboard operation
kafka-connect-storage-common copied to clipboard

Published 20 hours ago • verified publisher icon confluentinc

Upgrade parquet to latest version (1.13.1)

Open BDeus opened this issue 1 year ago • 1 comments

A PR already exist on 1.13.0

  • https://github.com/confluentinc/kafka-connect-storage-common/pull/315

Nice to have:

  • zstd without hadoop library https://issues.apache.org/jira/browse/PARQUET-1866

Need to test that this regression is fixed with 1.13.1:

  • https://github.com/confluentinc/kafka-connect-storage-common/pull/273
  • https://github.com/confluentinc/kafka-connect-storage-common/pull/305

BDeus avatar Jan 30 '24 15:01 BDeus

I test it, same issue than before

java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/lib/output/FileOutputFormat

the issue is in org.apache.hadoop.thirdparty in hadoop-shaded-* that used hadoop library related to parquet 1.11

BDeus avatar Jan 30 '24 17:01 BDeus