osdatahub icon indicating copy to clipboard operation
osdatahub copied to clipboard

Add a delay in the Feature API fetch loop to avoid 429 errors

Open ian-astun opened this issue 2 years ago • 1 comments

If you ask for a reasonable amount of data (say a county) python on my machine can exceed the (undocumented?) rate limit on even a relatively slow machine with a relatively slow broadband. So it would be useful to add a sleep step (preferably user configurable) to slow down the requests to prevent it just stopping.

ian-astun avatar May 30 '23 15:05 ian-astun

Not a bad idea... but an even nicer solution would be to detect 429 responses and then sleep, so that you go as fast as you can until you hit the limit.

mattlovett avatar Sep 08 '23 15:09 mattlovett