elementary icon indicating copy to clipboard operation
elementary copied to clipboard

Huge metadata folders in dbt-trino with Iceberg

Open mattiamatrix opened this issue 10 months ago • 0 comments

Describe the bug

Hello, I used Elementary with dbt-trino. Elementary tables are stored in S3 as iceberg tables.

Elementary has been running for a while, and the storage keeps growing due to the Iceberg metadata tables, which keep growing.

Image

Trino now supports

write.metadata.delete-after-commit.enabled = true
write.metadata.previous-versions-max = 10
  • https://github.com/trinodb/trino/pull/24306

--

What is the advised approach to maintaining elementary iceberg tables?Should elementary do something, or are users expected to handle it on their own?

mattiamatrix avatar Feb 24 '25 17:02 mattiamatrix