Michael Hoffmann
Michael Hoffmann
I guess it's fetching block metadata on startup ( the gateway) does it stabilize eventually and is the querier less noisy?
Storage gw is also 0.35.0 right?
How many blocks do you have in object storage roughly? Is your compactor working well?
The confusing thing about this to me is that ruler should not ask the querier to answer queries, it should answer from its local tsdb really. Querier is only needed...
I think the Thanos engine has an optimizer that sets the same matchers on both sides of the binary expression. Maybe that is broken for that query somehow. Are you...
@irizzant Does it happen with an empty rules file for ruler too?
> @MichaHoffmann I use kube-prometheus-stack chart which creates a bunch of Prometheus rules but they're not empty Does it also happen with an empty rules file? If not ~ can...
At a guess they probably fetch a lot of data from storage gateway, burnrate sounds like a thing that fetches a month or so.
cc @verejoel - did you solve that issue? I faintly remember that we discussed it on slack
I think our ruler http client is missing a timeout