pytrends
pytrends copied to clipboard
GeneralMills
Connect Timeout issue with proxies
I'm trying to use proxies from this url: https://sslproxies.org/ to pull large amounts of data from google trends, but I'm getting an issue in this line of code:
pytrend = TrendReq(hl='en-US', tz=360, timeout=(10,25), proxies=list(proxies), retries=2, backoff_factor=0.1, requests_args={'verify':False})
the list of proxies is from scraping the above website.
This is the issue:
ConnectTimeout: HTTPSConnectionPool(host='trends.google.com', port=443): Max retries exceeded with url: /?geo=US (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001E16F5FB4C8>, 'Connection to 118.174.232.44 timed out. (connect timeout=10)'))
Please advise.
problem is that you are making "too many" requests in the same time, add some try-except + sleep machanism maybe...
Alright, the issue can be two things. One you don't have working proxies, you can fix this by using a proxy checker like https://hidemy.name/en/proxy-checker/. The second one is that for some reason the person that programmed this assumed you have to use HTTPS and for some reason, it wont work properly. So, to fix this, go to request.py for the library, and change all the instances from HTTPS to HTTP, that is don't change any that are full URIs. after that make sure that in the proxy list you have https:// and not http://
Hello @rahulramaswamy, did any of the solutions work ? I have the same problem.
Hello @rahulramaswamy, did any of the solutions work ? I have the same problem.
Yea just put the procies as http instrad of https and it will work perfectly fine