Prashant Singh

As per my understanding, I think the issue here is that we are trying to bind all partition specs to to current schema which will always not be possible. For...

Thanks @rdblue @kbendick @nastra @amogh-jahagirdar for your awesome reviews :) !!!

Added a pr for the fix : https://github.com/apache/iceberg/pull/5860

can you please try once with `AssumeRoleAwsClientFactory` and creating cross-account IAM role. ref : https://iceberg.apache.org/docs/latest/aws/#cross-account-and-cross-region-access for more details.

> however, i am getting the following error when trying to write data to the table using df.writeTo("ns_name.xxxxxx").append() I think this is happening because you haven't specified the catalog name...

>Caused by: java.lang.UnsupportedOperationException at org.apache.iceberg.relocated.com.google.common.collect.ImmutableMap.put(ImmutableMap.java:714) I think this is happening because Kryo is not able to serialize / de-serialize Immutable map (this comes from `properties` in S3FileIO). (ref : https://groups.google.com/g/kryo-users/c/KP2FiJHFTzs)....

I see now, why this would not work basically in iceberg we have relocated guava, from `com.google.common.collect.ImmutableMap` to `org.apache.iceberg.relocated.com.google.common.collect.ImmutableMap` the serializer for guava in https://github.com/magro/kryo-serializers, won't work now as they...

We have faced similar issues before, for ex : https://github.com/apache/iceberg/pull/549/files I think the fix was to use to use java collection instead https://github.com/apache/iceberg/pull/546, which needs to be made at iceberg...

> i can create a custom serializer for org.apache.iceberg.relocated.com.google.common.collect.ImmutableMap using the code you provided above I think attempting custom serializer as temp workaround should be fair, iceberg has point releases...

@jessiedanwang I am not sure how kryo serializer comes to play here, are your spark confs overriden somewhere else ? Ideally if you don't specify any serializer by default i...