loki
loki copied to clipboard
grafana
Remove references to groupcache in config & metrics
Replace those references with a generic name like in-process cache (this gives us leverage to change the underlying technology later if we need to).
I guess it is a breaking change so maybe we don't do this as part of this work but if the idea is that the user (for now) would use the fifo cache as their 'in-process' cache for the index cache, should we also remove references to the 'fifo cache' as an option and instead replace those with 'in process'?
So basically when the user selects 'in process', we under the hood make the determination of whether its groupcache or fifo cache - the technology choice and the associated complexity is hidden from the user?
Interesting question @09jvilla... Maybe in-process cache is not the best terminology for us to use here (cc @kavirajk since you're working on this). I think we do still want to distinguish between fifo (localised) and groupcache (distributed), and I'm not sure we can make the decision for the user.
Maybe we go for the terms in-process-distributed and in-process-localized, and deprecate (but still support) fifocache?
Maybe we go for the terms in-process-distributed and in-process-localized
Seems ok with me, but maybe lets shop it around with the team? Naming is hard I know, but its also sticky. If we get a few other opinions we can at least flag any obvious gotchas.
deprecate (but still support) fifocache?
Should we maybe wait to transition from fifocache to in-process-localized until we're ready to move in-process-distributed out of experimental?
Stepping back, let's remember that the target user here is running Loki in simple-scalable mode, and doesn't want to have to configure any sort of cache, or run an external cache. So our aim should be making is as little effort as possible for a user to deploy Loki and get the caching they need with no additional configuration or deploying of extra services.
Ideally we can think about how we can use defaults or the common config to ensure that we select the correct in-process caches (whether distributed or localized) for each of our different caches (index, chunk, and query). In the end, there will be a lot of users that do not care at all about the details of what cache we use and whether its distributed or not. I agree some folks will want/need to tune, but lets figure out how to make it so that a basic setup would not require that and in the basic setup cache the user just 'gets caching' with no effort required (ideally, they don't even know the caching exists - we just do it for them and all they see is they get good query performance).
Damn naming is hard indeed :). My personal preference would be just calling it embeded-cache with flag distributed: true|false. Whether it's distributed or localized is still the implementation details IMO.
Other similar options could be
memory-cachelocal-cache
@09jvilla @dannykopping wdyt?
@kavirajk -- Wanna take this to our internal slack to get a wider set of opinions? I'm not sure I have enough technical depth to give a useful opinion here. It might be helpful to see a sample snippet for each of the caches (index, chunk, query) would look like with this proposal, but that might just be me asking due to lack of context.
https://raintank-corp.slack.com/archives/C03RVMAPQFL/p1659509932182539
more recent thread on how the config might look.
Should be resolved by: https://github.com/grafana/loki/pull/6821
@kavirajk -- is this right to link this PR here as resolving this issue?
At this point, its been so long that I've lost context on this issue, but I think the answer is "yes we can close it." Going to close this for now, and Kavi can reopen if he thinks we still need it.