pytrends icon indicating copy to clipboard operation
pytrends copied to clipboard

It broke again

Open dtaubaso opened this issue 1 year ago • 57 comments

I think that Google Trends is having the same issue it had past july/august, I'm getting lots of 429 and the embed code is broken on the website. Anyone having the same problems?

dtaubaso avatar Oct 19 '23 21:10 dtaubaso

Oh god, I thought I was the only one. Trying to get YouTube API Data for research and then 429s are plastered all across my screen. I really hope this gets resolved ASAP.

ImNotAProCoder avatar Oct 19 '23 22:10 ImNotAProCoder

I have the same bug, l have change the IP proxies and it works, but it failed get code of 429 today; Now l have no idea

zhajingwen avatar Oct 20 '23 06:10 zhajingwen

Same here

Helldez avatar Oct 20 '23 08:10 Helldez

nothing works even used proxies

praburamWAPKA avatar Oct 20 '23 14:10 praburamWAPKA

This time its even for a longer period not working. Last time the issue disappeared after a few days. It has been already two weeks since this issues are piling up.

image Orange = Sucess, Blue = Failed

Raidus avatar Oct 21 '23 07:10 Raidus

Any idea when it will be resolved?

praburamWAPKA avatar Oct 21 '23 15:10 praburamWAPKA

This time its even for a longer period not working. Last time the issue disappeared after a few days. It has been already two weeks since this issues are piling up.

image Orange = Sucess, Blue = Failed

You got any alternative solution?

praburamWAPKA avatar Oct 21 '23 16:10 praburamWAPKA

it seems that the guys at serp api have solved it

Helldez avatar Oct 21 '23 17:10 Helldez

Any idea when it will be resolved?

The origin of the problem is caused by Google: the hability to embed a trends graph is broken, so Pytrends API doesn't work Since this is not an official Google api, and we don't know if Google cares that much about its trends, we can't know when will this be solved.

dtaubaso avatar Oct 21 '23 17:10 dtaubaso

it seems that the guys at serp api have solved it

it seems that the guys at serp api have solved it

How you knew 🥹 how they fixed

praburamWAPKA avatar Oct 21 '23 17:10 praburamWAPKA

I think we can raise concern about the downtime in embeds... May be they will take a look

praburamWAPKA avatar Oct 21 '23 17:10 praburamWAPKA

it's already happened lately and it lasted more than a month, I think we should think about alternative scraping with selenium

Helldez avatar Oct 21 '23 17:10 Helldez

it seems that the guys at serp api have solved it

How you knew 🥹 how they fixed

because their searches work even with periods of less than a week

Helldez avatar Oct 21 '23 17:10 Helldez

it's already happened lately and it lasted more than a month, I think we should think about alternative scraping with selenium

By using cookies? That too temporary 🥹

praburamWAPKA avatar Oct 21 '23 17:10 praburamWAPKA

It works for me if I use a timeframe of today 1-m but it fails if I try now 1-d. I wonder if the arguments have changed for trends in the last 7 days?

mccoydj1 avatar Oct 21 '23 20:10 mccoydj1

I confirm it doesn't work from the weekly timeframe down, like the last time it lasted more than a month in July

Helldez avatar Oct 21 '23 20:10 Helldez

do you have a python function for utilizing results from the serpapi? I have this but the format is kind of bad

def query_google_trends(api_key, query_terms, timeframe): params = { "engine": "google_trends", "q": query_terms, "timeframe": timeframe, "api_key": api_key }

response = requests.get("https://serpapi.com/search", params=params)

if response.status_code == 200:
    data = json.loads(response.text)

    interest_over_time = data['interest_over_time']['timeline_data']

    return pd.DataFrame(interest_over_time)

else:
    return f"Error: {response.status_code}"

mccoydj1 avatar Oct 21 '23 21:10 mccoydj1

3 months duration is working for me and i checked the serpapi it shows USER TYPE LEGIT. It should be USER TYPE SCRAPER for them. They are using something to apply this

praburamWAPKA avatar Oct 22 '23 02:10 praburamWAPKA

except pytrends.exceptions.TooManyRequestsError as e: AttributeError: module 'pytrends.exceptions' has no attribute 'TooManyRequestsError'.

praburamWAPKA avatar Oct 22 '23 02:10 praburamWAPKA

Do you think it would be useful to set the request with these categories to build this URL? https://trends.google.com/trends/api/explore?tz=420&req=%7B%22comparisonItem%22%3A%5B%7B%22keyword%22%3A%22Retail+sales%22%2C%22geo%22

Helldez avatar Oct 22 '23 17:10 Helldez

I'm also trying to retrieve data with DataForSeo but is broken as well, I'm getting lots of errors

dtaubaso avatar Oct 22 '23 20:10 dtaubaso

Try this It works: https://serpapi.com/playground?engine=google_trends&q=test&geo=US&date=now+7-d&no_cache=true

Helldez avatar Oct 22 '23 20:10 Helldez

I'm also trying to retrieve data with DataForSeo but is broken as well, I'm getting lots of errors

I tried DFS yesterday but it worked well from my side, which errors did you get?

chxlium avatar Oct 24 '23 07:10 chxlium

Lots and lots of blanks and it took forever each time

On Tue, Oct 24, 2023, 04:58 Chenxi Liu @.***> wrote:

I'm also trying to retrieve data with DataForSeo but is broken as well, I'm getting lots of errors

I tried DFS yesterday but it worked well from my side, which errors did you get?

— Reply to this email directly, view it on GitHub https://github.com/GeneralMills/pytrends/issues/602#issuecomment-1776711266, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL325MN5DYZXBMY2RA7CFKLYA5YJHAVCNFSM6AAAAAA6H3KT3GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZWG4YTCMRWGY . You are receiving this because you authored the thread.Message ID: @.***>

dtaubaso avatar Oct 24 '23 11:10 dtaubaso

I am wanting to know how are you utilizing this in automation to end up with too many requests error. Like you are searching for keywords in cities or a country as a whole or just wanted to see latest trends in searches every minute? Just found this repo and wanted to know your use cases. Long back, I have dealt with similar issues but with Google maps API, hence curious

gl2007 avatar Oct 24 '23 18:10 gl2007

Try this It works: https://serpapi.com/playground?engine=google_trends&q=test&geo=US&date=now+7-d&no_cache=true

SerpApi is a good alternative but it's limited results per month for free and then it's $50/month which is too much for me to pay as a student who's researching. Really hope this can get this fixed soon :/

ImNotAProCoder avatar Oct 24 '23 20:10 ImNotAProCoder

I am wanting to know how are you utilizing this in automation to end up with too many requests error. Like you are searching for keywords in cities or a country as a whole or just wanted to see latest trends in searches every minute? Just found this repo and wanted to know your use cases. Long back, I have dealt with similar issues but with Google maps API, hence curious

Save the keywords and loop. It with multiple retries

praburamWAPKA avatar Oct 25 '23 00:10 praburamWAPKA

That sometimes works

On Tue, Oct 24, 2023, 21:08 praburamWAPKA @.***> wrote:

I am wanting to know how are you utilizing this in automation to end up with too many requests error. Like you are searching for keywords in cities or a country as a whole or just wanted to see latest trends in searches every minute? Just found this repo and wanted to know your use cases. Long back, I have dealt with similar issues but with Google maps API, hence curious

Save the keywords and loop. It with multiple retries

— Reply to this email directly, view it on GitHub https://github.com/GeneralMills/pytrends/issues/602#issuecomment-1778258170, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL325MNIN2CQVOI3DDIB5C3YBBKBLAVCNFSM6AAAAAA6H3KT3GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZYGI2TQMJXGA . You are receiving this because you authored the thread.Message ID: @.***>

dtaubaso avatar Oct 25 '23 00:10 dtaubaso

Try this latest version from pypi.org pytrends. It has the latest version 4.9.2. See this thread for explanation.

gl2007 avatar Oct 25 '23 02:10 gl2007

Try this latest version from pypi.org pytrends. It has the latest version 4.9.2. See this thread for explanation.

Everyone using latest version only.

praburamWAPKA avatar Oct 25 '23 02:10 praburamWAPKA