feat: TTL for fuse table

[BohuTANG]

Summary

Segment Level TTL to expire the data。 optimize table t1 purge will:

1. check the TTL of the table
2. purge the segments and blocks if they are expired

[caibirdme]

any update for this feature?

[BohuTANG]

@caibirdme Hi, no update yet.

Since Databend uses object storage, which is cheaper than local disk, space isn't an issue, and there is a vacuum command(the storage space will reclaim if data is deleted or a table is dropped): https://docs.databend.com/sql/sql-commands/ddl/table/vacuum-table

Future plans to adjust this remain undecided.

[caibirdme]

in vacuum-table doc, there's a setting DATA_RETENTION_TIME_IN_DAYS , is it a global setting? How can I make this setting vary from table to table? And is this user's task to run vaccum table routinely?

[TCeason]

in vacuum-table doc, there's a setting DATA_RETENTION_TIME_IN_DAYS , is it a global setting? How can I make this setting vary from table to table? And is this user's task to run vaccum table routinely?

Yes it's a global setting now. Maybe cc @lichuang can provide help

[BohuTANG]

in vacuum-table doc, there's a setting DATA_RETENTION_TIME_IN_DAYS , is it a global setting? How can I make this setting vary from table to table? And is this user's task to run vaccum table routinely?

We are planning to add this option to table.