sentry icon indicating copy to clipboard operation
sentry copied to clipboard

fix(alerts): Clean up querysubscription and snubaquery when alert is deleted

Open ceorourke opened this issue 1 year ago • 2 comments

We have a huge number of events on https://sentry.sentry.io/issues/2382970109 which happens when an alert rule has been deleted but the associated QuerySubscription and SnubaQuery are not deleted. This PR cleans up those rows when we find that the alert rule is missing. I left the metric so we can see it drop and even out (I expect it'll drop since we won't be repeatedly receiving subscription updates for a rule that doesn't exist anymore, and then even out at a much lower rate when we are just receiving an update right after a rule is deleted). Once it evens out we could move this logic to the deletions flow for the AlertRule model, but first we need to remove all the old stuff.

Closes https://sentry.sentry.io/issues/2382970109

ceorourke avatar Aug 07 '24 23:08 ceorourke

@wedamija lmk if I'm doing something silly here

ceorourke avatar Aug 07 '24 23:08 ceorourke

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 78.24%. Comparing base (819be93) to head (bf777a2). Report is 60 commits behind head on master.

:white_check_mark: All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #75787   +/-   ##
=======================================
  Coverage   78.24%   78.24%           
=======================================
  Files        6817     6816    -1     
  Lines      303640   303678   +38     
  Branches    52226    52235    +9     
=======================================
+ Hits       237571   237610   +39     
+ Misses      59685    59682    -3     
- Partials     6384     6386    +2     
Files Coverage Δ
src/sentry/incidents/subscription_processor.py 91.96% <100.00%> (+0.01%) :arrow_up:
src/sentry/snuba/tasks.py 93.91% <100.00%> (+0.05%) :arrow_up:

... and 28 files with indirect coverage changes

codecov[bot] avatar Aug 08 '24 01:08 codecov[bot]

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

  • ‼️ ProtectedError: ("Cannot delete some instances of model 'SnubaQuery' because they are referenced through protecte... sentry.snuba.tasks.delete_subscription_from_snuba View Issue
  • ‼️ SnubaQuery.DoesNotExist: SnubaQuery matching query does not exist. sentry.snuba.tasks.delete_subscription_from_snuba View Issue

Did you find this useful? React with a 👍 or 👎

sentry[bot] avatar Aug 08 '24 20:08 sentry[bot]

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

  • ‼️ ProtectedError: ("Cannot delete some instances of model 'SnubaQuery' because they are referenced through protecte... sentry.snuba.tasks.delete_subscription_from_snuba View Issue

Did you find this useful? React with a 👍 or 👎

Haha, damnit, you need to run the query with AlertRule.objects_with_snapshots.filter

wedamija avatar Aug 08 '24 20:08 wedamija