cortex icon indicating copy to clipboard operation
cortex copied to clipboard

Document blocks storage per-tenant retention

Open pracucci opened this issue 3 years ago • 1 comments

The PR https://github.com/cortexproject/cortex/pull/3879 introduced per-tenant retention for the blocks storage but we currently don't have any documentation about it. We should mention it in the doc.

Ideas:

  • We could mention it in docs/blocks-storage/compactor.template
  • We could have a "Retention" guide under docs/blocks-storage mentioning it + mention that a bucket policy along with -querier.max-query-lookback=<retention period + small delta> may be another alternative but wouldn't provide a per-tenant granularity

pracucci avatar Mar 08 '21 14:03 pracucci

Hey @pracucci, I am currently setting up the retention policy for my cortex, and I am interested in contributing to this issue. I am think that we could do the following:

  1. Add the global retention config in docs/blocks-storage with -querier.max-query-lookback=<retention period + small delta>.
  2. Add per tenant config in runtime-config, add a sectioncompactor & block storage like what we have for Ingester, Distributor & Querier limits.
  3. Link 2 from 1, so that user can navigate to per-tenant config if they are interested in that.

WDYT?

jiachengxu avatar Jan 10 '22 04:01 jiachengxu