rill icon indicating copy to clipboard operation
rill copied to clipboard

Support `TTL` for ingested data

Open k-anshul opened this issue 1 year ago • 1 comments

Given now we support incremental models with splits, it will be good to also allow users to set a TTL on their data. A common use case will be to load data for last X days for serving dashboards. Request raised by @harshmohta

k-anshul avatar Sep 24 '24 10:09 k-anshul

For clickhouse this is possible with native support, for duckdb need to build support

k-anshul avatar Sep 24 '24 11:09 k-anshul

Given that we can leverage ClickHouse's native support here, I think we can close this issue for now. In the future, we may want to support retention policies (probably a SQL filter that matches rows to drop) generically, which would also solve this problem for DuckDB.

begelundmuller avatar Dec 10 '24 14:12 begelundmuller