iceberg icon indicating copy to clipboard operation
iceberg copied to clipboard

Published 20 hours ago • verified publisher icon apache

Use Java collections in AwsProperties to fix Kryo serialization.

Open jfz opened this issue 2 years ago • 1 comments

jfz avatar Sep 20 '22 19:09 jfz

This seems very similar to: https://github.com/apache/iceberg/pull/546

Error while running Spark 3.3 + Iceberg 0.14: Caused by: com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException Serialization trace: s3BucketToAccessPointMapping (org.apache.iceberg.aws.AwsProperties) xxxxx io (org.apache.iceberg.spark.source.SerializableTableWithSize) at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:101) at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:508) at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575) at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:79)

jfz avatar Sep 20 '22 21:09 jfz

Thanks @jfz , looks good to me!

Thank you for reviewing @szehon-ho !

jfz avatar Nov 05 '22 05:11 jfz

Merged, thanks @jfz , @nastra @rdblue for review

szehon-ho avatar Nov 05 '22 07:11 szehon-ho