sherlock
sherlock copied to clipboard
Use retries/back-off for retrying when rate limited.
This adds retries with backoffs to requests, this has lead to now no longer getting false positives when rate limited and fixes a site.
Should this be a new version number?
Well, maybe version bump when the other PR with the test fixes is in. This may lead to some excess extra requests but trying this n large amounts of accounts, it all worked out well. Maybe some CLI options should be considered or seeing if there's a way to make the Retry object work in a way where we can provide lambda so it can check if status_code is being checked, if so, if it's that specific status code, don't retry?
Well, any way, this works well.