Rate limits occasionally lead to errors.

[JacksonBarker]

trafficstars

Summary

Occasionally, rate limits result in errors instead of warnings. In this case the request is not rescheduled.

Reproduction Steps

For testing, a simple bot is created that will await message.delete() when on_message(message) is called. Users will produce simulated spam in an attempt to overwhelm the API rate limit.

Minimal Reproducible Code

@client.event
    async def on_message(message: discord.Message):
        await message.delete()

Expected Results

The bot will re-schedule all DELETE requests that are missed while the bot was rate limited WARNING discord.http We are being rate limited. DELETE https://discord.com/api/v10/channels/XXXXXXXXXX/messages/XXXXXXXXXX responded with 429. Retrying in 0.XX seconds.

Actual Results

For some few messages, instead of deleting the message, the following error message is displayed

ERROR    discord.client Ignoring exception in on_message
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/discord/client.py", line 441, in _run_event
     await coro(*args, **kwargs)
   File "//./bot.py", line 20, in on_message
     await message.delete()
   File "/usr/local/lib/python3.10/site-packages/discord/message.py", line 841, in delete
     await self._state.http.delete_message(self.channel.id, self.id)
   File "/usr/local/lib/python3.10/site-packages/discord/http.py", line 759, in request
     raise HTTPException(response, data)
 discord.errors.HTTPException: 429 Too Many Requests (error code: 0): You are being rate limited.

Intents

discord.Intents.all()

System Information

- Python v3.10.11-final
- discord.py v2.2.3-final
- aiohttp v3.8.4
- system info: Linux 5.15.0-71-generic #78-Ubuntu SMP Tue Apr 18 09:00:29 UTC 2023

Checklist

• [X] I have searched the open issues for duplicates.
• [X] I have shown the entire traceback, if possible.
• [X] I have removed my token from display, if visible.

Additional Context

No response

[mikeshardmind]

This sounds like a bad idea. If you're consistently reaching the only code path that can have this happen, you're getting a 429 without at least some of the ratelimit info, see: https://github.com/Rapptz/discord.py/blob/e870bb1335e3f824c83a40df4ea9b17f215fde63/discord/http.py#L677-L680

And it's likely a Cloudflare 429 from other times people have hit this. There's no good way to reschedule this even if it did make sense at that point, and it's better to error out when you've got a pathological case that got picked up by cloudflare than to keep trying anyhow.

[JacksonBarker]

My concern is that I was able to trigger this condition with a single user (On a different network/ IP). Using a bot for moderation, it is safe to assume that even a server with a dozen or so online users would overwhelm the bot easily. That doesn't make sense, how are other unverified bots able to avoid this issue?

[DevilXD]

I'd like to point out that the usual way of getting 429s is not necessarily a Cloudflare ban, but rather getting five 429s in a row, five being the maximum amount of retries the request method goes for, before giving up and raising the last error code as the exception. It's relatively easy to run into that if you have a lot of requests waiting their turn in rate limit to hit the same endpoint, especially if your code is as terrible as the minimum reproducible you've provided. Taking a closer look at the traceback, this part only confirms that: (...)/http.py", line 759, in request(...).

https://github.com/Rapptz/discord.py/blob/e870bb1335e3f824c83a40df4ea9b17f215fde63/discord/http.py#L754-L759

If you really need to delete messages as soon as they're sent, then you're much better off setting up some kind of a queue where all sent messages get added into, and then a separate task will pick messages from the queue to delete them. That way, you're waiting on at most one rate limit per operation, instead of everything ending up in rate limit at once. An extra optimization would be to pool messages from a single channel for a short while after the first message is sent, and then use the bulk-delete endpoint to delete all of them at once.

I'd say this is not a bug. The wrapper cannot prevent you doing stupid things with the Discord API. You're the one who's supposed to structure your code in a way that'll throttle the amount of hits to a particular endpoint, without relying on the Discord API to do this throttling via rate limits for you. That'll usually only lead to an actual Cloudflare ban, for consistently trying to go over the rate limits imposed by the API.

[JacksonBarker]

The bug is rate limit errors in a wrapper that claims to feature rate limit handling. I am aware of more efficient or logical ways to do what is shown in the MRC.

[mikeshardmind]

The stuff right above the lines which were linked have the actual ratelimit handling:

discord.py does handle ratelimits, it just doesn't sit around forever if for some reason you're exceeding them in ways the application can't see the cause of, or if you're doing something that will never Stop being ratelimited (like attempting to delete every message)

Some common causes of this range from being on a shared IP to having multiple instances of the bot running (and therefore exhausting ratelimits outside of the view of discord.py), to absolutely flooding with requests like attempting to delete every message you see. The latter of which, no amount of ratelimit following will save you on and discord.py is still doing the right thing to abort trying (Not hammering the remote API by ignoring that you're still limited)

[JacksonBarker]

Attempting to delete every message was the minimum amount of code to reproduce the issue, the real bot's purpose was regex'ing addresses and other malicious messages. Regardless, the same day as opening the issue, I solved my problem by just queuing requests. I chose to keep the issue open because my expectation would be that the wrapper would implement its own queue to prevent rate limiting instead of just throwing an error.

If something like that is out of scope, then I'll just close the issue. Like I said, I solved my problem already over a year ago.