iceberg icon indicating copy to clipboard operation
iceberg copied to clipboard

Docs: Update defaults for distribution mode

Open szehon-ho opened this issue 7 months ago • 0 comments

In recent releases of Iceberg-Spark, there have been changes to Spark defaults for distribution mode, by changes like: https://github.com/apache/iceberg/pull/7637 (and previous ones).

I have seen many questions regarding why an extra shuffle is included in writes, and it is often this change in default. One source of confusion I have seen is the doc mentioning table property write.distribution-mode which says default is NONE, so this PR tries to add more to here to explain this.

szehon-ho avatar Jun 27 '24 21:06 szehon-ho