mute_time_intervals should be inherited to sub routes
What did you do? I set up a route to mute all warning alert outside work days, and several sub routes to route these alerts to different receivers.
What did you expect to see? I was expecting this alert could be silenced in all sub routes
What did you see instead? Under which circumstances? the alert was sent regardless of the mute intervel setting Environment
-
System information:
insert output of
uname -srmhere -
Alertmanager version:
insert output of
alertmanager --versionhere (repeat for each alertmanager version in your cluster, if relevant to the issue) v0.23.0 -
Prometheus version:
insert output of
prometheus --versionhere (repeat for each prometheus version in your cluster, if relevant to the issue) -
Alertmanager configuration file:
insert configuration here
- Prometheus configuration file:
insert configuration here (if relevant to the issue)
- Logs:
insert Prometheus and Alertmanager logs relevant to the issue here
I'm aware the document said:
# Times when the route should be muted. These must match the name of a
# mute time interval defined in the mute_time_intervals section.
# Additionally, the root node cannot have any mute times.
# When a route is muted it will not send any notifications, but
# otherwise acts normally (including ending the route-matching process
# if the `continue` option is not set.)
mute_time_intervals:
[ - [<string>](https://www.prometheus.io/docs/alerting/latest/configuration/#string) ...]
but I think these rules should be inherited, this can make the config shorter and easier to read, otherwise I have to set mute time for every single route.
I also found the current behavior to be inconvenient. And the documentation says that children inherit optional parameters from the parents in the routing tree, so this behavior is also not obvious.
I haven't checked this, but it sounds like a bug to me - /cc @benridley @simonpasquier
Agreed that it doesn't look very consistent and I see the point of having the same mute times being applied to all subroutes .
One reason though to explain the current behavior: unlike group_interval et al., an empty mute_time_intervals is valid so if time intervals were inherited by child routes, it would be hard to distinguish between "apply from the parent" and "don't apply from the parent".
Thanks @juliusv - I've opened #3538 to fix this. It is indeed a bug, I forgot to check for empty config on child routes before overwriting parent config, so it overwrote the parent config every time.