Use Sentry issue priority to specify `severity` on outgoing events
Problem Statement
My team uses Sentry to create incidents on Pagerduty when a new event is added to an issue. Sometimes there are noisy issues that come from error logs or thrown exceptions, and we want to be able to control the noise without outright archiving the issue.
I set up Event Orchestration in Pagerduty that sets the urgency of the incident based on the severity sent by Sentry. When the severity is error, we page the on-call. When the severity is warning, we file an incident but do not page the oncall.
I'd like to be able to set a severity level based on the Sentry Issue priority, rather than the underlying event level. This way, if a noisy issue pages the on-call, they would be able to set the Issue Priority in Sentry to Medium. Then they will still receive incidents, but won't be woken up in the middle of the night from a page.
Sentry currently only allows me to filter based off of event level, not Issue Priority.
Solution Brainstorm
I'd like to be able to set a filter based off of Issue Priority. This way, I could set up an alert that declares:
- When an event is captured by Sentry, and the Issue's Priority is High
- Send a notification to Pagerduty with Error severity
And another alert that could declare:
- When an event is captured by Sentry, and the Issue's Priority is Medium
- Send a notification to Pagerduty with Warning severity.
And so on for Low priority issues.
Product Area
Alerts
Assigning to @getsentry/support for routing ⏲️
Routing to @getsentry/product-owners-alerts for triage ⏲️
@tiny-crab - thank you for filing this feature idea! We're making some bigger changes to how alerts are configured, and filtering based on priority is on our list of changes to add in
@rachrwang That's great to hear. I'm looking forward to seeing the new features. What's the timeline for this?
Hey @tiny-crab, I think as a very rough estimate I'd say EOY or beginning of next year.
Okay, that's good to hear. Excited for that to be built and released. How will I know when it's complete?
We have some larger UI changes planned, so it should be very noticeable when it takes effect 😅
Cool, looking forward to seeing the new changes.
Just checking in, is this feature still likely to be added by EOY? I'm scoping out some work to work around this in the meantime, but curious how your roadmap looks in the next couple of months.
@tiny-crab - our changes will start rolling out in the first half of next year. Thanks!
@tiny-crab we have added these new priority filters that might meet your needs:
@ceorourke I just checked those out, thanks for the tip.
What does it mean by marks in those two filters? Does the priority need to change in order to trigger an alert using that event type?
If I'm not mistaken, I'm more looking for a filter in the IF section of the alert configuration. Since I want to manually change the Sentry issue priority based on noise, and have it sent at a lower severity to Pagerduty.
If I'm not mistaken, I'm more looking for a filter in the IF section of the alert configuration. Since I want to manually change the Sentry issue priority based on noise, and have it sent at a lower severity to PagerDuty.
Yeah, you're correct that you'd want to have the if section create a filter to then send that to pager duty. The marks info is basically saying "If an issue's priority is set or changed to high, then trigger the following actions"
As @rachrwang mentioned, we're actively working on a feature to make this waayyy more robust. We'll have a configuration to allow you to define the issue priority as you described. That way you could just create issues that are lower priority for your team as low priority issues. We'll also introduce the if filter that you described as a way to handle the severity levels. We think the combination of features will allow you to really hone into when / how to be alerted about issues.
@saponifi3d Thanks! That's exciting to hear. @rachrwang had mentioned these features would be rolling out in H1 2025... any updates on how soon that would be? :D
@tiny-crab - we don't have any updates yet on timing, though we will definitely follow up as we have more details to share!
Just checking in to see if there’s any update on this @rachrwang
We're still actively working on this, it should be released in the new few months but I can't make promises regarding the timeline.