mimir icon indicating copy to clipboard operation
mimir copied to clipboard

Make query-frontend cache TTL configurable, and increase the default.

Open colega opened this issue 2 years ago • 5 comments

Is your feature request related to a problem? Please describe.

query-frontend caches results, but it sets a TTL of 7d to those results:

https://github.com/grafana/mimir/blob/cac269d4837e168b3b3f17b7e2e56cdacd79c4c7/pkg/frontend/querymiddleware/split_and_cache.go#L37-L38

If you're caching a year-long query, it means that you'll have to recalculate it next week again, which is undesirable.

Describe the solution you'd like

Make the TTL configurlable, allow no TTL at all (Memcache is LRU, why caring about TTL?)

Note that the maximum TTL for Memcache appears to be 30d.

colega avatar Dec 30 '22 14:12 colega

Make the TTL configurlable, allow no TTL at all (Memcache is LRU, why caring about TTL?)

Please double check memcached evicts items with no TTL too.

pracucci avatar Jan 05 '23 13:01 pracucci

Hello, I would like to work on this issue. Is there anything I should know before starting (Besides the contributing guidelines)? Thank you

Gabrielopesantos avatar Feb 04 '23 21:02 Gabrielopesantos

Please double check memcached evicts items with no TTL too.

I would love someone to investigate ☝️ this, cause we should set no TTL at all if there's no downside.

pracucci avatar Feb 06 '23 12:02 pracucci

From what I read in the documentation, non expired items (an expiration time of 0 or some time in the future), are evicted when there's no memory available to allocate new insertions. An expiration time of 0 means that no automatic expiration is done.

With that, I am not seeing any downside in not having a TTL at all as all evictions are treated equally.

Gabrielopesantos avatar Feb 06 '23 22:02 Gabrielopesantos

From what I read in the documentation, non expired items (an expiration time of 0 or some time in the future), are evicted when there's no memory available to allocate new insertions. An expiration time of 0 means that no automatic expiration is done.

It's also possible to configure Redis like this so the default would be portable if/when Redis support is added.

56quarters avatar Feb 06 '23 23:02 56quarters