flickr-download icon indicating copy to clipboard operation
flickr-download copied to clipboard

Throttling to avoid API query limits

Open ddffnn opened this issue 11 months ago • 1 comments

Flickr limits API usage, and after using flickr-download I appear to have gone over the limit because progress has slowed to a crawl. Looking at my app stats, I was over 3600 queries per hour for 3 hours, so I'm assuming my app is limited (at least temporarily).

I still have more pictures to download, so I'm hoping I can resume at full speed after waiting a while. That being said, I assume I'll run into the same situation again without throttling queries from flickr-download. I think a one query per second rate is the obvious choice.

Flickr api documentation:

Limits: Since the Flickr API is quite easy to use, it's also quite easy to abuse, which threatens all services relying on the Flickr API. To help prevent this, we limit the access to the API per key. If your application stays under 3600 queries per hour across the whole key (which means the aggregate of all the users of your integration), you'll be fine. If we detect abuse on your key, we will need to expire the key, or turn it off, in order to preserve the Flickr API functionality for others (including us!). We also track usage on other factors as well to ensure no API user abuses the system.

ddffnn avatar Mar 08 '24 04:03 ddffnn

Oh, good find. Yes, we should probably add an option to throttle requests

beaufour avatar Mar 16 '24 12:03 beaufour