Tumbr rate limit being hit (for default API key)

[OmgImAlexis]

Since you bundle an API key it's being used too much, maybe allow the user to supply their own key?

{"meta":{"status":429,"msg":"Limit Exceeded"},"response":[]}

[Hrxn]

Should already work, try to put your key in rip.properties.

[OmgImAlexis]

There really needs to be some kind of documentation or just remove the default key as it's obviously being used too much. Tumblr does rate limit depending on IP as well as hits so if they've worked out this key is being bundled with this then it'll just keep getting less and less allowed hits before being limited like this.

[rautamiekka]

^ Is there anything provided by Tumblr RipMe could benefit from ?

[OmgImAlexis]

@rautamiekka not too sure I understand your question.

[rautamiekka]

^ Tumblr-provided docs on API.

[OmgImAlexis]

All that needs to be done is the default key needs to be removed and there should be a box asking for the API key when the user types in .tumblr.com. Nothing else can really be done.

[metaprime]

Documentation: https://www.tumblr.com/docs/en/api/v2

I was under the impression that this API key was a registered API key for the application, in which case they would expect it to be bundled in an application (I think). I think it makes sense to have a default key and maybe prompt the user to register their own key when they try to use the default one. I think if the settings contain no API key then the default one will be used rather than prompting the user, but I'd have to check on this.

[OmgImAlexis]

I've had the same thing happen to my own key before as I run a Chrome extension that had a lot of users. After a while Tumblr just ends up lowering the limits further and further. It's even worse for things scraping the API.

Another thing you could do is just use the api v1 without a key.

Example: http://reblogalert.tumblr.com/api/read/json?debug=1&start=0 Docs: https://www.tumblr.com/docs/en/api/v1

[metaprime]

I'm guessing there's a reason we stopped using the keyless API v1...

[OmgImAlexis]

@metaprime looks like you've always used API v2 https://github.com/4pr0n/ripme/commits/master/src/main/java/com/rarchives/ripme/ripper/rippers/TumblrRipper.java

[metaprime]

@OmgImAlexis I guess it's worth investigating the keyless API but something tells me each user would pretty quickly go over the rate limit.

[OmgImAlexis]

The rate limit per IP for the AP v1 would be a lot higher than the rate limit for everyone using the API v2.

[metaprime]

Fair enough... Although if someone wants to provide an API v2 key wouldn't that be better? I acknowledge that these two concerns can coexist.

[OmgImAlexis]

Not really, the API v1 was depreciated but Tumblr still prefers that over v2 for scraping. I'll see if I can find the email thread I have with one of their devs but they basically confirmed this for one of my own projects as it causes less stress on their servers.

[metaprime]

@OmgImAlexis excellent to know!