alertmanager
alertmanager copied to clipboard
Proposal: improve silence API
What did you do? I add silence for too noisy alerts already being open incidents in our incident management system. I added an identifier (e.g. ticket nr) from the incident management system as author/ createdBy or to the comment.
What did you expect to see? I like the incident management system to be able to query silences for a specific createdBy content or even for pattern in the comment to check if the silence still exists (as it needs to be cleaned up before closing an incident).
What did you see instead? Under which circumstances? Silences can be filtered by matching labels only.
Environment
not relevant
Proposal
on /silences GET
route, there should be implemented additional filter parameters for the createdBy and comment field not for the matchers only.
Additionally, a filter for active silences may make sense.
Alternative There could also be done separate full-fledged proposal proposing an additional "labels" data structure on silences, which would avoid any regex searches on the comment field. (not to be mixed up with the alert labels, here those would be labels set by the user / receiving systems on the silence entry).
Goals
- make it possible to receive a filtered list of Silences on silence-specific fields like createdBy or comment
Audience Everyone using alertmanager together with an incident mgmt system and going into automation.
Possible action plan
- adapt openapi spec
- implement additional filters
- optional: add gui functionality
Resources We might be able to spend some resources on it, for sure on testing side, to support a timeline of having this capability merged until end of July.
References
- alertmanager openapi v2 spec: https://github.com/prometheus/alertmanager/blob/main/api/v2/openapi.yaml
I start lightweight and will enrich this issue with results from discussion in #[email protected]