feat: Add support of multiple kind of cache for relabeling components

[pbailhache]

PR Description

Related to proposal introduced by https://github.com/grafana/alloy/pull/1600.

This is a (working) draft for this feature.

Which issue(s) this PR fixes

Notes to the Reviewer

PR Checklist

• [ ] CHANGELOG.md updated
• [ ] Documentation added
• [ ] Tests updated
• [ ] Config converters updated

[mattdurham]

Before we get to far down this path, I have a few longer term concerns about using this while we still have the handling of a single metric at a time. In an ideal world the Appender a batch based interface so we can batch the cache requests but thats a bigger lift.

[wilfriedroset]

Thank you @mattdurham for your review and feedbacks. regarding your concerns

about using this while we still have the handling of a single metric at a time. In an ideal world the Appender a batch based interface so we can batch the cache requests but thats a bigger lift.

Do you think it would be possible to split the two needs? First we introduce the external caches via this PR, then after we take time to design and implement the batching to optimize further the performance.

As you have pointed, batching the requests will require more work. The reason why I'm suggesting that is because the current inmemory cache is the default and will continue to work as it is now with the same behavior/performance. The performance improvement added by the refactoring will benefit the external caches only so there is no risks. When I'm saying later, this is something we are willing to contribute but we might require help due to the complexity.

WDYT?

[mattdurham]

Really want to see some benchmarks with using redis/memcache, use something like https://golang.testcontainers.org/modules/redis/ with relabel and a few thousand signals.

[pbailhache]

I made a bench branch, you can see the commit here : https://github.com/pbailhache/alloy/commit/95098e04fff2b249910eb67e2fc4d639f8f8e38a

Here are the result of the benchmark on my laptop, running with dockertest.

It's a bit better with a running docker container outside of the benchmark (around 65000ns/op for redis). I used this benchmark to optimize the encode/decode for redis & memcached, switching from gob to json as it a bit quicker for small structs.

Here is the 3 flamegraphs for each backend, as we can see, the Get calls to the redis/memcached are what makes it slow. LRU :

Redis

Memcached

Here I'm still using the deskit/cache package, I chose to use it because I did not want to re-implement the whole cache clients system. Do you think it's better to not use it here, I could implement a simpler version of each client and bench that, but I don't think the difference will be huge.

[mattdurham]

Actually we are already using some dskit structs in a few places, will review your imports. We recently found an issue with exemplars that will likely require changes to the appender/appender interface that will batch the samples which should significantly improve the viability of this. I would hold off until we get that out, which will likely be in two weeks.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}