prometheus icon indicating copy to clipboard operation
prometheus copied to clipboard

Add support for defining series and alert limits individually in rule groups

Open vjsamuel opened this issue 11 months ago • 2 comments

This PR attempts to define rule group parameters for series and alert limits so that individual limits can be applied to recording and alerting rules respectively.

Alert managers typically suffer when too many alerts are emitted. Rule groups can have a mix of both recording and alerting rules and it is beneficial to end users to be able to define each limit separately.

This PR takes the global limit as-is if neither is defined. If either alert or series limits are defined, then they are honored first.

Ex:

groups:
- name: example
  rules:
  - alert: InstanceDown
    expr: count(prometheus_http_requests_total) by (handler) > 0
    for: 1m
    labels:
      severity: page
    annotations:
      summary: "Instance {{ $labels.instance }} down"
      description: "{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 5 minutes."
  alert_limit: 1

vjsamuel avatar Mar 12 '24 07:03 vjsamuel