prometheus
Resolve timeout per alert
Can we get feature to set different resolve_timeout per alert based on matchers?
It would be cool because we can have different alerts that request different configurations.
For example for alert that will alert me on high memory usage i would like to set higher resolve_timeout so when i receive alert i want to be sure that memory usage have stabilized, but for server down i want alert to resolve as soon as server is back up.
Hi, thanks for reaching out! Are you using Prometheus to generate your alerts? If so, have you considered using keep_firing_for instead? It sounds like it might map better to your needs. See the docs here: https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/#rule
Alertmanager's resolve_timeout is really intended for clients that don't properly set EndsAt on alerts they send to Alertmanager. Under normal circumstances, Prometheus sets EndsAt (which means resolve_timeout doesn't actually do anything).
@Spaceman1701 Hi, yes i am using prometheus to generate alerts.
This should help, because as i see keep_firing_for is per alert setting.
Thank you for response!