sentry icon indicating copy to clipboard operation
sentry copied to clipboard

Provide a status text containing the reason for HTTP 429

Open danielkhan opened this issue 3 years ago • 11 comments

Problem Statement

Our backend returns an HTTP error 429 when some rate limit/quota of our public API is reached. There is no more information about the reason.

Solution Brainstorm

Return a status text that contains information about which rate limit was reached.

danielkhan avatar Jul 27 '22 10:07 danielkhan

Routing to @getsentry/team-web-sdk-backend for triage. ⏲️

getsentry-release avatar Jul 27 '22 17:07 getsentry-release

Routing to @getsentry/ingest for triage. ⏲️

getsentry-release avatar Jul 28 '22 10:07 getsentry-release

@danielkhan can you elaborate what you're seeing and provide full response headers and data?

untitaker avatar Jul 28 '22 10:07 untitaker

The 429 is returned from the API when a rate limit is met and it lacks an error text that explains the real reason. I don't have an example, just users requesting help on this.

danielkhan avatar Jul 29 '22 09:07 danielkhan

Routing to @getsentry/ecosystem for triage. ⏲️

getsentry-release avatar Jul 29 '22 10:07 getsentry-release

in an offline discussion it turned out this is about API calls to Sentry (/api/0/), not store API calls (/api/n/). This probably warrants broader discussion in #discuss-api but as far as assigning teams for triage goes, I suppose either S&S or ecosystem is it?

untitaker avatar Jul 29 '22 10:07 untitaker

I can improve the message in the error, but in the meantime, rate limit usage details are in the headers.

AniketDas-Tekky avatar Aug 01 '22 17:08 AniketDas-Tekky

@AniketDas-Tekky can you provide an example of what this would look like? If we document this (if it isn't yet) and let support know, this might be good enough.

danielkhan avatar Aug 04 '22 08:08 danielkhan

Seems like this is actually documented at https://docs.sentry.io/api/ratelimits/

On Thu, Aug 4, 2022 at 10:42 AM Daniel Khan @.***> wrote:

@AniketDas-Tekky https://github.com/AniketDas-Tekky can you provide an example of what this would look like? If we document this (if it isn't yet) and let support know, this might be good enough.

— Reply to this email directly, view it on GitHub https://github.com/getsentry/sentry/issues/37103#issuecomment-1204949476, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGMPRMT7IVNLVV4WRIWCCLVXN65TANCNFSM54Y7RW6Q . You are receiving this because you are on a team that was mentioned.Message ID: @.***>

untitaker avatar Aug 04 '22 10:08 untitaker

This seems to apply to regular rate limits but what if a customer is over quota?

danielkhan avatar Aug 04 '22 16:08 danielkhan

there is no quota for API calls like this

untitaker avatar Aug 05 '22 16:08 untitaker

What if they consumed all transactions?

danielkhan avatar Aug 08 '22 07:08 danielkhan

@danielkhan We have two APIs: The APIs sdks talk to to submit data (/api/123/{store,envelope}/ where 123 is the project id, served by Relay), and the APIs the web UI (and scripts cobbled together by customers) uses to query data (/api/0/).

the rate limits concerning any path under /api/0/ have nothing to do with what the user pays for and are only there to protect our infrastructure from greater harm. It's not possible to submit any crash reports using any endpoint starting with /api/0/.

If you run out of transactions, errors or attachment quota, we will start rate limiting requests under /api/123/.

Which API are we talking about? Typically when ppl talk about "the API" they mean /api/0, if they mean /api/123 it's typically called "ingest API" or "store endpoint(s)"

untitaker avatar Aug 08 '22 10:08 untitaker

@untitaker, this was the original discussion where a user uses Node with tracing and sees 429. That's all we know and a symptom of the problem is that we can't tell the user where to look next.

We also had another request where a user would see the errors in the JavaScript console of the browser. Again, we either don't report the cause properly or support does not know which next step they should suggest to the user.

I thought providing an error text in the payload whenever a 429 occurs would be a pragmatic solution.

danielkhan avatar Aug 08 '22 10:08 danielkhan

okay, I'm unassigning ecosystem from this for now because this customer request is about sdk requests, not "the API" as we typically call the endpoints we use for querying data.

untitaker avatar Aug 08 '22 12:08 untitaker

Routing to @getsentry/ingest for triage. ⏲️

getsentry-release avatar Aug 08 '22 12:08 getsentry-release

Just had a customer report a mysterious 429 with no way to debug. Admin/stats_v2 has nothing so presumably load balancer.

Reopening as the original request has not been really resolved (fix just added same generic message + link to docs regardless of reason).

There is no more information about the reason. Return a status text that contains information about which rate limit was reached.

realkosty avatar Mar 05 '24 19:03 realkosty