elementary
elementary copied to clipboard
Huge metadata folders in dbt-trino with Iceberg
Describe the bug
Hello, I used Elementary with dbt-trino. Elementary tables are stored in S3 as iceberg tables.
Elementary has been running for a while, and the storage keeps growing due to the Iceberg metadata tables, which keep growing.
Trino now supports
write.metadata.delete-after-commit.enabled = true
write.metadata.previous-versions-max = 10
- https://github.com/trinodb/trino/pull/24306
--
What is the advised approach to maintaining elementary iceberg tables?Should elementary do something, or are users expected to handle it on their own?