Search for Session Replays are rate limited in the UI
Environment
SaaS (https://sentry.io/)
Steps to Reproduce
- Search for session replays (with many results)
specific example linked in shadow ticket
Expected Result
Search loads a list
Actual Result
Product Area
Replays
Link
No response
DSN
No response
Version
No response
┆Issue is synchronized with this Jira Improvement by Unito
Auto-routing to @getsentry/product-owners-replays for triage ⏲️
We seem to have a quite short rate limit going here. When I tried to repro, I couldn't for a few tries. Data loaded properly, I believe, 3x full page refresh. Then I got rate limited.
Thanks for raising, we'll dig into what's going on.
Hi Bruno, do you have any updates? This is still affecting our whole engineering and support teams, as we're rarely able to search for session replays.
I'm happy to record a loom video reproducing the issue, if that helps. We're keen to have this issue resolved, as it heavily affects our productivity. Thanks.
Thanks for writing in @rafalzv7labs. This is a p0 for us and we'll be looking into to it asap.
We identified the rate limit is coming from Snuba: https://github.com/getsentry/sentry/blob/97a906cd8b69a52dd2c56928438e897e033a55ad/src/sentry/utils/snuba.py#L338
Still looking at values and other conditions
We increased limits on our end and I couldn't repro being rate limited after a few refreshes. Could you verify things work as you expected?
If not, please give us more information about the usage patterns. Is there a script calling into our APIs on your end? Does this happen every N hours? Some times of the day? etc
For the tag querying issue we need to apply an index to the tags field and update our query optimizer to use the index when possible. The way we'll index the field requires a new materialized column and will take 90 days to back fill. Though we may use it opportunistically before then by introspecting the time range of the query.
I haven't seen this issue come up again. Let me know if customers are still seeing errors. The tags index is still in the process of rolling out but it should complete in the next ~45 days. The tags index should be working for anyone with a stats-period of 30 days or lower.