grafana-prometheus-alertmanager-datasource icon indicating copy to clipboard operation
grafana-prometheus-alertmanager-datasource copied to clipboard

Published 20 hours ago • verified publisher icon camptocamp

Alert filtering by labels doesn't work

Open mfuhrmann opened this issue 6 years ago • 12 comments

  • Grafana 4.6.3
  • Prometheus Alertmanager Plugin 0.0.4

I have some labels as you can see here: grafik

When I try a filter expression like context="PSP-T" still all alarms are visisble in Grafana. Also a filter like instance="IP-ADDRESS" which should only hit one single alarm is not working. I always get all alarms.

mfuhrmann avatar Apr 12 '18 06:04 mfuhrmann

It probably helps if you post an screenshot of how exactly your are querying the alertmanager (metrics section of the configuration).

SPFZ avatar Apr 12 '18 10:04 SPFZ

grafik

mfuhrmann avatar Apr 12 '18 11:04 mfuhrmann

Looks about right for me, are you 100% sure the context label is present in the alert? Can you put it in the legend to check?

SPFZ avatar Apr 12 '18 11:04 SPFZ

Not sure what you mean. This one https://user-images.githubusercontent.com/4995222/38625039-e3c96562-3da9-11e8-8315-5e3e209b6d60.png is a screenshot of alertmanager which shows that I have the context labels, for example context="PSP-T". Unfortunately the second screenshot https://user-images.githubusercontent.com/4995222/38673880-deb64154-3e52-11e8-9cb6-cf8b254705fd.png is an other example which uses context="IT". Sorry, but I didn't have an other one at this moment.

mfuhrmann avatar Apr 12 '18 11:04 mfuhrmann

I am also having the same issue. I can display all my current alerts on the table. If one has a product table of test and i put product='test' in to the query box, it doesn't seem to do anything... in fact i cant get any query to work. saying that the lable field works and its still great being able to display all my alerts...

davidferdinand avatar May 25 '18 20:05 davidferdinand

Could you open the "query inspector" and check if your label is set under response -> data -> 0 -> labels?

cryptobioz avatar Jun 02 '18 13:06 cryptobioz

yep, its under that, but it just doesn't seem to do anything...

davidferdinand avatar Jun 03 '18 17:06 davidferdinand

grafik

mfuhrmann avatar Jun 06 '18 12:06 mfuhrmann

Same problem here. Given that using template variables is not supported, this bug kills the only case where I could use Grafana for alerts.

federico-razzoli avatar Nov 21 '18 09:11 federico-razzoli

@santec Which version of Grafana and of the datasource are you using? Could you share the request created by the datasource and sent to the alertmanager (value of url in the Grafana's query inspector)?

cryptobioz avatar Jan 21 '19 14:01 cryptobioz

Meanwhile it' Grafana 5.3.4 and 0.0.7 in my case. And it still does not filter. URL in my case: url:"https://server.domain.com/alert-manager//api/v1/alerts?silenced=false&inhibited=false&filter=context%3D%22IT%22%0A"

mfuhrmann avatar Mar 04 '19 14:03 mfuhrmann

Since I'm using 6.2.x and the current version of this plugin (which is still the same), it works for me.

mfuhrmann avatar Sep 12 '19 11:09 mfuhrmann