Bryan Keller

Cool, thanks for the feedback and consideration

Thanks for pointing out memory settings @dongjoon-hyun , the goal of this PR is to bring more consistent memory usage behavior between YARN and k8s rather than to solve for...

You mentioned there is a reason for the different modules, can you explain the reason k8s is configured to use a fixed heap size and YARN does not?

I don't follow how a YARN PR will help? AFAIK, the executor heap is [set by Spark](https://github.com/apache/spark/blob/07061f1a07a96f59ae42c9df6110eb784d2f3dab/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala#L132)

FYI, I removed the delta writers along with upsert and CDC related code.

I believe I've addressed all comments, LMK if there is anything else.

> What's expected to be included in the next kafka-connect PR? > > Asking b/c I'm doing some work on implementing zombie-fencing in tabular/iceberg-kafka-connect currently and we may want to...

I was planning on merging this, unless someone wants to give more feedback, cc @fqaiser94 @danielcweeks

I believe when we add https://github.com/apache/iceberg/pull/11313 you should be able to accomplish mapping topics to tables. Also I think this PR isn't complete, the new config isn't being used.

Thanks @yornstei, good to know you found this useful. I was wondering if you had an opinion on https://github.com/apache/iceberg/pull/11623?