pytrends icon indicating copy to clipboard operation
pytrends copied to clipboard

How to avoid HTTP status 429 (too many requests)?

Open Rombersoft opened this issue 2 years ago • 4 comments

Who can share experiences about problems with HttpStatus 429 (Too many requests)? I noticed that the developer thought about using the proxy server. Does it exist some statistics about what I must do in order to be banned by Google Corporation? Public API is missing and how much is legal to use google back end in order to show google trends results on my own site???

Rombersoft avatar Oct 14 '22 21:10 Rombersoft

what function are you using?

rheophile10 avatar Oct 31 '22 16:10 rheophile10

what function are you using?

I think it does not mean. You execute a request and get 429 status

Rombersoft avatar Nov 08 '22 18:11 Rombersoft

@Rombersoft I was trying to understand how pytrends produces that result. But if your question is in general what to do with HTTPStatus 429 and how to get around that, stackoverflow is probably a better forum for you.

rheophile10 avatar Nov 08 '22 19:11 rheophile10

I think I figured it out. By default, pytrends doesn't supply a user agent, so it uses the default one. This causes the request to be blocked.

I propose allowing a user to pass a user-agent in via the requests_args argument.

Edit: https://github.com/GeneralMills/pytrends/pull/553/commits/18f230dd69c2cd89c203c967ac139a8218975f4d adds it


You can then add a user agent like this:

from pytrends.request import TrendReq

requests_args = {
    'headers': {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36'
    }
}

# Only need to run this once, the rest of requests will use the same session.
pytrend = TrendReq(requests_args=requests_args)

# Continue normally
...

xenova avatar Dec 04 '22 22:12 xenova

Fixed by #553.

emlazzarin avatar Dec 26 '22 18:12 emlazzarin