sentry icon indicating copy to clipboard operation
sentry copied to clipboard

Alerts failing to notify custom integration

Open kerenkhatiwada opened this issue 9 months ago • 6 comments

Environment

SaaS (https://sentry.io/)

Steps to Reproduce

  1. Create a custom integration
  2. Create an alert with an action of "Send a notification via [custom integration]", example

When the alert is triggered, no event is sent to the custom integration. However, when using the "Send Test Notification" an event is received. This seems to have stopped working for the customer on April 29th.

Expected Result

When the alert is triggered, an event is sent to the custom integration.

Actual Result

When the alert is triggered, no event is sent to the custom integration.

Product Area

Alerts

Link

https://org2-w7.sentry.io/alerts/rules/react/15159560/details/

DSN

No response

Version

No response

┆Issue is synchronized with this Jira Improvement by Unito

kerenkhatiwada avatar May 03 '24 16:05 kerenkhatiwada

Assigning to @getsentry/support for routing ⏲️

getsantry[bot] avatar May 03 '24 16:05 getsantry[bot]

Routing to @getsentry/product-owners-alerts for triage ⏲️

getsantry[bot] avatar May 03 '24 16:05 getsantry[bot]

Hi there, according to the custom integration's dashboard Sentry is still sending webhooks to the site.

Is it possible that the webhook URL is expired or has reached a limit request? I believe that the webhook site the integration is using has an expiration date on the free tier.

schew2381 avatar May 06 '24 23:05 schew2381

Hey @schew2381, Wehhooks are sent when an issue/errors gets created, but none when the alert is triggered. Event type: event_alert.triggered.

It doesn't look like the webhook site has a limit request or expiration date yet. You can access this specific URL here.

The customer (in the JIra ticket) is encountering the same and not using the webhook site.

kerenkhatiwada avatar May 06 '24 23:05 kerenkhatiwada

Ok I did some digging and on the customer's custom integration, and it appears that at some point the custom integration settings for webhooks was turned off, and the webhook was unsubscribed from receiving events.

Screenshot 2024-05-07 at 11 32 37 AM

This can happen if the webhook receives many timeouts in 24 hrs, as detailed by this changelog.

If the webhook is resubscribed to receiving events, does that fix the issue at hand?

schew2381 avatar May 07 '24 18:05 schew2381

Hey @schew2381, the customers attempted to resubscribe the webhook, but they are still not able to receive events of Event type: event_alert.triggered. They do see however see event types of issue.create and error.create. Is there anything else they can test to receive "Alert Rule Action"?

kerenkhatiwada avatar May 09 '24 16:05 kerenkhatiwada

Routing to @getsentry/product-owners-settings-integrations for triage ⏲️

getsantry[bot] avatar May 10 '24 21:05 getsantry[bot]

Routing to @getsentry/product-owners-alerts for triage ⏲️

getsantry[bot] avatar May 10 '24 21:05 getsantry[bot]

@ceorourke would you be able to add insight here?

nhsiehgit avatar May 10 '24 22:05 nhsiehgit

The app in question isn't subscribed to any webhooks still, all the boxes are unchecked.

ceorourke avatar May 10 '24 22:05 ceorourke

Closed as resubscribing to webhooks fixed the problem.

schew2381 avatar May 31 '24 19:05 schew2381

Reopening as I cannot read english and was mistaken

schew2381 avatar May 31 '24 20:05 schew2381