Discord-Scraper icon indicating copy to clipboard operation
Discord-Scraper copied to clipboard

Performance Improvements

Open Dracovian opened this issue 4 years ago • 0 comments

The requests library isn't asynchronous, so that means I'll have to implement aiohttp in future releases. I'll also have to look into the ratelimiting issue but Discord was gracious enough to provide us developers with the following details regarding ratelimiting:

X-RateLimit-Global	= Returned when HTTP 429 is thrown.
X-RateLimit-Limit	= The number of requests that can be made.
X-RateLimit-Remaining	= The number of remaining requests that can be made.
X-RateLimit-Reset	= The epoch time at which the rate limit resets.
X-RateLimit-Reset-After	= Total time (in seconds) of when the current rate limit bucket will reset.
X-RateLimit-Bucket	= A unique string denoting the rate limit being encountered.

So I'll just have to keep the X-RateLimit-Remaining and X-RateLimit-Reset headers in mind when using aiohttp to avoid issues regarding ratelimiting.

Until I figure out the way to properly manage asynchronous data in Python, I won't be going much further than using aiohttp for added performance for a while.

Dracovian avatar Apr 23 '20 10:04 Dracovian