thanos
thanos copied to clipboard
thanos-io
Incorrect regex label match when using `|` with an empty option
Thanos, Prometheus and Golang version used: Thanos: v0.22.0 Prometheus: v2.28.1
Object Storage Provider: S3
What happened:
Using a Thanos Store Gateway as the data source, prometheus_build_info{doesnotexist=~"whocares|"} (note the tailing | on that regex) returns no series.
What you expected to happen:
I expected the Store Gateway to return the same series as prometheus_build_info{doesnotexist=""}, which is what a Prometheus data source does.
How to reproduce it (as minimally and precisely as possible):
Write a query with label=~"value|" on a metric without a series with label="value".
Use Store Filtering in the Query UI to select either a Prometheus instance or a Store Gateway.
Example query:
prometheus_build_info{doesnotexist=~"whocares|"}
Full logs to relevant components: Not applicable, I don't think.
Anything else we need to know:
Basically Prometheus appears to interpret {label=~"value|"} as either {label=value} or {label=""} (label does not exist). I expected Thanos to do the same.
Let me know if you need anymore info.
I was able to reproduce this on 0.22.0. Interesting bug. Thanks for the report! Help welcome.
The reason is that our optimization on regex is not exactly the same as Prometheus' one.
Our implementation: https://github.com/thanos-io/thanos/blob/4276b50564d745d28a9f6ce3e92d19d6fd50c46f/pkg/store/bucket.go#L1763.
Prometheus: https://github.com/prometheus/prometheus/blob/b1ed4a0a663d0c62526312311c7529471abbc565/tsdb/querier.go#L221
As {doesnotexist=~"whocares|"} matches the empty string "", Prometheus will go for an inverse posting match. However, Thanos doesn't check whether the empty string is matched or not so the logic is different.
I would like to work on this issue, could you please guide me about how should I proceed.
I suppose a good start would be to add failing tests. A suitable location seems to be the e2e tests: https://github.com/thanos-io/thanos/tree/main/test/e2e.
Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}
Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.
Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.
