Research Sentry's alerting and webhooks functionality in order to generate a specification for a Sentry Feature Health provider

[matthewelwell]

[emyller]

While Sentry's support for feature flags is still a beta product, [tracked] evaluated feature flags are not [yet] part of any payload Sentry will output as webhooks. Now since our Feature Health webhook receiver expects the suspect/affected feature flags in the input, Sentry is unable to contribute to FH when issues arise, as of now.

I wonder if we could achieve this by other means, or if any Sentry competitor will offer this.

@matthewelwell Is there any action we could take from here?

[khvn26]

@emyller Have you read about Sentry's integration platform? If we opt to develop a Sentry app for Feature Health, here's supposedly a bunch of ways to link issues and issue/metric alerts to external resources. Can we explore that?

At the very least, I'd look for a way to read a feature name from issue tags as they can be set from Sentry SDK.

[emyller]

@khvn26 My research comes exactly from integration platform. 😉 I have failed though to link the relevant Sentry docs to my comment above. Fixed.

[matthewelwell]

On a discussion with @emyller, there are a couple of outstanding points:

1. The error webhook doesn't include any contextual information about any flags
2. We are unsure what the future functionality might look like if we were to add our own custom handler for the change tracking webhook, instead of the generic one.

For point 2, we need to reach out to Sentry.

---

The ideal integration might be more like: we count the number of given sentry errors against each feature, and define thresholds as to what defines an unhealthy feature in Flagsmith?

In order to achieve this, we would need:

1. To implement evaluation tracking, or have customers use OpenFeature
2. Have Sentry include contextual information based on the evaluation tracking data in the error webhooks

For the latter, we need to reach out to Sentry.

Next steps:

• [x] Submit docs PR to Sentry (#5620) - could this be a way to also ask the 2 questions that we have for the Sentry team? (@emyller)
• [ ] Ask the Flagsmith / Polychrome team if we have any contacts at Sentry that we could direct our 2 questions to (@matthewelwell) - this is done, pending response from Sentry.