grequests icon indicating copy to clipboard operation
grequests copied to clipboard

Append wait time or rate for every thread due to server limits

Open latot opened this issue 2 years ago • 1 comments

Hi, actually some server has some limits, is hard to tell a server too much requests and in some ones the rate can be very low.

I think would be great be able to set a strategy to be able to handle this, some ideas:

  • Set a rate of request per second
  • wait some time after complete a request

I think would be pretty useful implement something like this.

Thx!

latot avatar Jun 07 '23 21:06 latot

I think this would be a very useful feature. I've looked at some existing implementations of rate-limiting and I was a bit overwhelmed by the number of different ways rate-limiting can be implemented.

It's not clear to me if any one rate limiting strategy stands out as more useful in the context of grequests than others. I suppose the rate limiting a user would want grequests to use depends on how the server implements rate limiting.

In any case, I'd like to figure this out and get it implemented.

spyoungtech avatar Jun 08 '23 00:06 spyoungtech