Anton Okolnychyi
Anton Okolnychyi
This PR reduces the serialization in `DeleteOrphanFilesSparkAction`.
What I see frequently is that people clone some examples but miss to update the metrics config. As a result, it points to fields that don't exist. I think we...
It should be safe to ignore files added by rewrite operations in `BaseOverwriteFiles` while checking whether concurrent appends may contain matching data (i.e. `conflictDetectionFilter`).
If we migrate existing tables to Iceberg and then continue writing to them, file locations are not consistent because of how we handle spaces in `PartitionSpec$partitionToPath`. For example, we have...
I am not sure we handle case-sensitivity correctly while binding fields in `PartitionSpec` and `SortOrder`.
We would need to design an API for users to put their logic for managing keys to support [modular encryption](https://github.com/apache/parquet-format/blob/master/Encryption.md) in Parquet. Also, we would need to propagate the encryption...
### What changes were proposed in this pull request? This PR contains new connector APIs for loading stored procedures per [discussed and voted](https://lists.apache.org/thread/w586jr53fxwk4pt9m94b413xyjr1v25m) SPIP tracked in [SPARK-44167](https://issues.apache.org/jira/browse/SPARK-44167). It is a...