badger "cannot use generic type ristretto.Cache[K z.Key, V any] without instantiation"

[carsonip]

update-beats https://github.com/elastic/apm-server/pull/14296 is pulling in latest beats (with badgerv4 dep after https://github.com/elastic/beats/pull/40939 ) which pulls in github.com/dgraph-io/ristretto v0.1.2-0.20240116140435-c67e07994f91 which contains a breaking change and compile error. See issue in ristretto https://github.com/dgraph-io/ristretto/issues/394 and https://github.com/dgraph-io/ristretto/issues/386

 Error: #25 16.09 /go/pkg/mod/github.com/dgraph-io/badger/[email protected]/table/table.go:79:14: cannot use generic type ristretto.Cache[K z.Key, V any] without instantiation
Error: #25 16.09 /go/pkg/mod/github.com/dgraph-io/badger/[email protected]/table/table.go:80:14: cannot use generic type ristretto.Cache[K z.Key, V any] without instantiation

ristretto has cut a v0.2.0 release before the breaking change, and v1 after. But this does not fix our use case.

Potential solutions

1. APM server has to bump to badgerv4 (e.g. v4.3.1 so use ristretto v1). Work to update badger is tracked in https://github.com/elastic/apm-server/issues/11546.
2. Revert beats badgerv4 bump https://github.com/elastic/beats/pull/40939
3. I don't think there's a way to use both badgerv4 (via beats) and badgerv2 in apm-server at the same time as v1 ristretto will be picked over v0 as they share the same import path, and badgerv2 code will get a compile error. Unless we convince ristretto to release a v2 and make badgerv4 use it. See discussion.
4. Revert beats bump in apm-server

APM Server version (apm-server version): in PR branches e.g. https://github.com/elastic/apm-server/pull/14296

Provide logs (if relevant):

https://github.com/elastic/apm-server/actions/runs/11232054143/job/31222853639