Drastic drop in high frequency alerts

[boxinegmbh]

Self-Hosted Version

24.1.1

CPU Architecture

x86_64

Docker Version

25.0.0

Docker Compose Version

1.29.2

Steps to Reproduce

Upgrade from 23.3.1 to 24.1.1 Full docker volume backup was done Restore was successful

Expected Result

Uninterrupted functionality of the configured alerts.

Actual Result

After upgrade, steep decline in triggered alerts. Tried setting up the alerting behaviour again but no change observed

Alert settings

Event ID

No response

[azaslavsky]

Did you do the hard stop at 23.6.2 when upgrading?

[getsantry[bot]]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

[boxinegmbh]

We upgraded from v23.3.1 to v23.4.0 to manage the postgres database upgrade Then v23.4.0 to v23.9.1 and did the full docker volume backup Then v23.9.1 to v23.11.1 used hardware with recommended requirements Finally v23.11.1 to v24.1.1 when we saw this issue Performed the Recovery steps suggested in the docs https://develop.sentry.dev/self-hosted/troubleshooting/#recovery We did not perform the nuclear option since we wanted to recover all issues with no/minimal data loss What exactly is the Hard stop?

[hubertdeng123]

A hard stop is a version of self-hosted Sentry that must be upgraded to before continuing. Are there any logs in your web container that might be able to tell us what might be going on? Looks like alerts are not being triggered.

[boxinegmbh]

After investigation, seems like sentry was amplifying the impact of errors (impacting 50% of sessions), after the upgrade the drastic drop is observed. The number of errors did not drastically change before and after upgrade. Current assumption is the alerts based on the rate of sessions was wrong before upgrade to v24.1.1