trino
trino copied to clipboard
Iceberg-connector: Delete the oldest tracked version metadata files after commit
Description
When using the iceberg table, the xxx-metadata.json file is generated each time commit is executed. In the iceberg table, we can automatically clean the previous metadata file through configuration.
write.metadata.delete-after-commit.enabled = true
write.metadata.previous-versions-max = 10
However, the metadata file is not automatically cleaned in trino. A large number of metadata.json files are left on the hdfs.
Different from #20863 ,I have followed the configurations in iceberg (write.metadata.delete-after-commit.enabled and write.metadata.previous-version-max ). Instead of adding a new configuration, this keeps it compatible with iceberg.
Release notes
( x) Release notes are required, with the following suggested text:
## Iceberg connector
* Delete the oldest tracked version metadata files after commit. ({issue} #19582 ).