Michael Hoffmann comments

Results 224 comments of


                                            Michael Hoffmann

Ruler: Evaluations stop without errors or logs

Yeah I think that this at least should make this an error instead of a deadlock. We should take a timeout from config though, technically this is also a breaking...

Sidecar is significantly slower than the underlying Prometheus queries

This might very well be because we now force the store response to be sorted and we only can rely on that by resorting which forces us to buffer the...

Sidecar is significantly slower than the underlying Prometheus queries

Oh! Can you share the query or an anonymized version of it please? It sure sounds like the issue is in the engine then.

Sidecar is significantly slower than the underlying Prometheus queries

It shouldnt but the query ( if its range query ) might be affected by query frontend caching so for sake of reproducibility it would be good to use query...

promql: implement `integral()` function

How about monte-carlo integration - shouldnt that be somewhat robust towards missing samples?

query: regression in v0.36.0 - concurrent_gate_queries periodically reaching limit

Likely https://github.com/thanos-io/thanos/pull/7618 I guess! Can you try 0.36.1?

receive: does not cleanup old data on disk, but tries to load it on boot, causing OOM down the line

Sure - can you help figuring out what happens here? What version are you running? I think WAL should be managed by the embedded Prometheus libraries

receive: does not cleanup old data on disk, but tries to load it on boot, causing OOM down the line

Anything interesting in logs or metrics?

Receive: Endless loop of retried replication with capnproto and distributors

Do you use a ingester/router split? if you do that then you cannot create a loop in the resulting bipartite directed graph! I think thats the recommended way to not...

chore: Bump Rust version(s)

Do we have the secrets configured for this repository? I would add the actions then today