loki icon indicating copy to clipboard operation
loki copied to clipboard

Compactor: Make it possible to delete logs without setting `retention_enabled: true`

Open kavirajk opened this issue 1 year ago • 4 comments

Is your feature request related to a problem? Please describe. Currently you cannot have logs delete feature in compactor without setting retention_enabled: true. Meaning, you also have to set the retention_period to some artificial high values to avoid deleting logs accidentially.

Why this matters? Say you have exported logs from Grafana cloud to your own bucket. Now you just want to delete logs on demand via compactor (say running Loki SSD with target=backend on top of your own bucket), it's currently confusing to set correct config for compactor.

Basically, compactor deletion and retention is coupled right now and worth de-coupling it.

Describe the solution you'd like Make compactor delete endpoints work without needing to set retention related config

Describe alternatives you've considered live with confusing configs dependency with deletion and retention. Enabling retention and setting retention period to some artificial value to avoid deleting older logs seems non-intuitive.

Additional context Drive came after helping one of the cloud logs customer who is using Cloud logs exporter, facing this same problem

kavirajk avatar Nov 09 '23 10:11 kavirajk