alertmanager icon indicating copy to clipboard operation
alertmanager copied to clipboard

Filtering active silences via API does not work as expected

Open oalimerko opened this issue 2 years ago • 5 comments

What did you do?

I ran this API call for filtering all active silences in Alertmanager with severity="high" ,severity="high" is a match label

http://127.0.0.1:9093/api/v2/silences?silenced=false&active=true&filter=severity="high"

What did you expect to see?

I expect to see only silences with severity="high" which are active.I can get this via amtool

What did you see instead? Under which circumstances? I can see the whole silences ,active and expired ,with severity="high"

Environment Win 10

  • System information:

MSYS_NT-10.0-19044 3.3.3-341.x86_64 x86_64

  • Alertmanager version:

alertmanager-0.21.0.windows

  • Prometheus version:

Prometheus is not used.Only a dummy AM instance for testing purpose,all alerts are created manually in AM via amtool

  • Alertmanager configuration file:
global:
  resolve_timeout: 5m

route:
  group_by: ['alertname']
  group_wait: 10s
  group_interval: 10s
  repeat_interval: 1h
  receiver: 'web.hook'
receivers:
- name: 'web.hook'
  webhook_configs:
  - url: 'http://127.0.0.1:5001/'
inhibit_rules:
  - source_match:
      severity: 'critical'
    target_match:
      severity: 'warning'
    equal: ['alertname', 'dev', 'instance']

oalimerko avatar May 23 '22 07:05 oalimerko

Can confirm. I queried active silences but it returned all silences:

curl -H "Content-Type: application/json" "http://localhost/api/v2/silences?active=true"
[{"id":"someid","status":{"state":"expired"},"updatedAt":"2022-08-24T05:26:25.447Z","comment":"blablabla"},
{"id":"some other id","status":{"state":"expired"},"updatedAt":"2022-08-24T05:25:52.845Z","comment":"blabla something else"}]

I am using prom/alertmanager:v0.24.0 image.

tommysitehost avatar Aug 24 '22 05:08 tommysitehost

It doesn't look like the openapi spec does specify anything else than a filter parameter which only can operate on matchers.

https://github.com/prometheus/alertmanager/blob/main/api/v2/openapi.yaml

Tontaube avatar May 31 '23 11:05 Tontaube

Some news ? Purhaps tag component/api and component/silences ?

tguenneguez avatar Oct 10 '23 12:10 tguenneguez

faced the same problem on the next version of the alertmanager

» ~  curl -sS '.../api/v2/status' | jq .   
{
  "cluster": {
    "peers": [],
    "status": "disabled"
  },
....
  "uptime": "2023-11-17T12:53:32.714Z",
  "versionInfo": {
    "branch": "HEAD",
    "buildDate": "20220325-09:31:33",
    "buildUser": "root@265f14f5c6fc",
    "goVersion": "go1.17.8",
    "revision": "f484b17fa3c583ed1b2c8bbcec20ba1db2aa5f11",
    "version": "0.24.0"
  }
}

mgerasimchuk avatar Dec 01 '23 14:12 mgerasimchuk

2 years and no response. Just wanted to report I am seeing the same/similar issue in version 27. I run the active=true&silenced=true query and receive alerts which have no items in the silencedBy list. It's unclear to me if this is a bug or a feature. If it is the latter, I think more examples or better documentation would help to clear up confusion.

My issue: An alert has 3 states: active, suppressed, and unprocessed. So if I am filtering for active alerts that are silenced do I query for active=true&silenced=true, or do I query for active=false&silenced=true. I also don't think the defaults make sense, and don't reflect what is seen in the regular Alertmanager web UI. Silenced, inhibited, and unprocessed filter all default to true, but when you use the web UI they default to false (have to check a box to see silenced alerts). Is there a reason for this difference?

**edited to specify I am facing this issue on the /alerts endpoint, not the /silences endpoint.

cbryant42 avatar Apr 03 '24 15:04 cbryant42