oncall icon indicating copy to clipboard operation
oncall copied to clipboard

Customise alert limits for team and integration

Open tommysitehost opened this issue 2 years ago • 2 comments

We should be able to set/update alert limits for team and integration in the settings interface: https://github.com/grafana/oncall/blob/dev/engine/apps/integrations/mixins/ratelimit_mixin.py#L18

I experienced stuck alerts in oncall. Seems like this is what happened:

  • “firing” alerts were issues
  • Alert Group were registered
  • Rate-Limit engaged
  • “resolve” alerts were issued
  • “resolve” alerts were not recorded because of a rate-limit
  • Rate-Limit disengaged
  • Alert Group stayed in the “Firing” state forever

Should resolve alerts get rate limited? IMO I think we should only rate limit firing alerts?

tommysitehost avatar Nov 04 '22 00:11 tommysitehost