loki icon indicating copy to clipboard operation
loki copied to clipboard

Custom variable in grafana with Stream selector {$label_name=~"$label_value"} is not correct works

Open Vitaliy155 opened this issue 1 year ago • 1 comments

Describe the bug Grafana dashboard variable works incorrect where variable definition as {$label_name=~"$label_value"}

To Reproduce

  1. Create 3 variables in the Grafana dashboard: label_name - Query type: Label names; label_value - Query type: Label values, Label: {$label_name}; loglevel - Query type: Label values, Label: loglevel, stream selector: {$label_name=~“$label_value”}

  2. Save and go to dashboard.

  3. Choose Label name:host, Label value: , level: one of the available levels. In my case it's ERROR. There is no level ERROR ... but should be!

Expected behavior Should be available value from custom variables related to label_value. As workaround, you can see label_value: "ERROR" without custom variable if choose: label_name: "loglevel" and label_value: "ERROR"

Environment: Grafana: 10.4.3 Loki:3.0.0

Screenshots, Promtail config, or terminal output 1 2 33 label_loglevel label_name label_value

### Tasks

Vitaliy155 avatar May 17 '24 13:05 Vitaliy155

Same issue - I have used stream selectors forever to filter logs from labels with variables dynamically, and they seem to get blanked out in the UI when saved - and ignored altogether when filtering additional nested variables.

I have been using this since Loki was released:

label_values(compose_project)	

service
label_values({compose_project=~"$project"}, compose_service)	

container
label_values({compose_project=~"$project", compose_service=~"$service"}, container_name)	

host
label_values({compose_service=~"$service", container_name=~"$container"}, host)	

lux4rd0 avatar Jul 01 '24 01:07 lux4rd0

This Bug basicly breaks chained variables when using loki as a datasouce :( grafik

itsek avatar Aug 01 '24 07:08 itsek

We are stuck to Grafana 11.0.0 because of this issue. I have tried to upgrade to any newer versions (11.1 or 11.2) and it breaks dashboard chained variables every time.

broferek avatar Sep 13 '24 09:09 broferek

I tried to filter the labels using the stream selector and it seems that it didn't have any impact on the query. If i choose specific value for variable which i use in the stream selector, it doesn't have any impact on the query result, it still provide all results.

LibiKorol avatar Dec 08 '24 06:12 LibiKorol

FYI @LibiKorol I managed to update to the latest Grafana version after first upgrading to the latest Loki 3.3 version

broferek avatar Dec 08 '24 19:12 broferek

Hi, this seems to have been fixed for the most part, however we still get some non-existent labels, which doesn't break the dashboards but some of the labels give values which don't have data for the chosen time range, without any visible pattern. We tested with grafana's 11.1 up to 11.5.2 and Loki versions 3.1.1 and 3.3.2 and the problem still exists. With grafana versions 11.0.0 and 10.4, there's no problem.

KaloyanHr avatar Feb 27 '25 08:02 KaloyanHr