gtrendsR
gtrendsR copied to clipboard
PMassicotte
Status code 429
Im trying to run an old script where I get the gtrends of 10 terms with a sleep time of 3-5 seconds. However, I can barely make 4 queries before I get the 429 status code.
Was there an update in the search limits? If so, how are they now?
Here's a repex of what im trying to do:
terms <- c("calderas + compra", "spa", "Cosmetology & Beauty Professionals",
"equipos + electricos", "ars humano", "Insurance", "caribe group + santo domingo",
"precios + cobre", "importadora + madera", "Electronics & Electrical"
)
searches <- map(terms, gtrends)
You also my want to deal with queries that do not return something:
> for (i in terms) {
+ print(i)
+
+ try(gtrends(i))
+ Sys.sleep(1)
+ }
[1] "calderas + compra"
[1] "spa"
[1] "Cosmetology & Beauty Professionals"
Error : No data returned by the query. Consider changing search parameters.
[1] "equipos + electricos"
[1] "ars humano"
[1] "Insurance"
[1] "caribe group + santo domingo"
[1] "precios + cobre"
[1] "importadora + madera"
[1] "Electronics & Electrical"
I'm running into a similar issue with this code.
g <- gtrends(keyword = "food stamps", geo = "US-MN", time = "today 12-m", gprop = "web")$interest_over_time
I run this daily and it has worked mostly well over the past two years. Recently I've been getting this error:
Status code was not 200. Returned status code:429
I've tried the code again with more popular keywords like "NFL" and still get the error.
However after a few tries it just works with no changes to the code.
Any thoughts on why that might be happening?
Interesting, it seems that 429 is related to too many requests:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429
- Something changed on Google's side?
- Check https://github.com/GeneralMills/pytrends if some similar issues pop-up in the next few days.
- https://github.com/GeneralMills/pytrends/issues/492
I'm also seeing this. I've tried multiple queries of different terms from home network, from remotely behind a corporate VPN, and from a custom EC2 instance of R and experience the same issues.
I've run into this a lot. My workaround has been to budget ample time and make multiple runs with a VPN, changing endpoints each run, and then collating the results into a master file. I'd love to figure out a way, with a service like NordVPN, where I can simply change endpoints per request the moment a request hits a 429, but I'm not sure they have an API that supports that.
Hello - FYI - I found this - https://github.com/trendecon/trendecon/blob/master/R/gtrends_with_backoff.R
Thanks to @karoliskoncevicius
This is still an issue to this day. I used multiple IP addresses via VPN as well as my own, and get the 429 error code even with the simplest .build_payload request. That means the "too many requests" thing is an illusionary error message, since it will just fail by default on the first try. The code I used worked fine for months until this week, and the only thing I did was install a newer python version (3.10.6 to be exact), as well as set up all modules freshly of course. Therefore it might be a fair guess that the pytrends module is simply incompatible with higher versions.
Hello, apparently, Google changed the portal and the API, can anyone confirm that the code as per today is working? I try a super simple script like this
library(gtrendsR)
trends <- gtrends("Biden",
gprop = 'web',
geo = 'US',
time = "2023-02-15 2023-04-14" )
I receive
Error in FUN(X[[i]], ...) :
Status code was not 200. Returned status code:429
for me the following works
gtrends("Biden",
gprop = 'web',
geo = 'US',
time = "2023-02-15 2023-04-14", onlyInterest = TRUE )
the problem is that the function related_topics or related_queries fails. This is imo because they are called to fast in succession. There simply needs to be a Sys.wait() between the calls. Then everything else should work as the usual. API hasn't changed imo.
API hasn't changed imo.
https://searchengineland.com/google-launches-new-google-trends-portal-394026 I thought it did, let me try again and thanks for your comments
I have the same issue her. gtrends() is no longer reliable. Sometimes everything works fine, but more often I get Status code was not 200. Returned status code:429. It used to work perfectly a month a go.
Sample code:
gtrends("NFL", geo="US", gprop = "web", time = "today+5-y", low_search_volume = T, onlyInterest = T)
I've come across the same issue with this. I used to get 429 errors using a similar process in Python, but it was reliable in R. I'm now getting 429 errors using gtrendsR
As I mentioned in my comment above, Google may have change some stuff in their API for the trends portal but it was not fully confirmed yet
for me the following works
gtrends("Biden", gprop = 'web', geo = 'US', time = "2023-02-15 2023-04-14", onlyInterest = TRUE )the problem is that the function related_topics or related_queries fails. This is imo because they are called to fast in succession. There simply needs to be a Sys.wait() between the calls. Then everything else should work as the usual. API hasn't changed imo.
For me this gives the same "Status code was not 200. Returned status code:429"
Same issue:
Error in FUN(X[[i]], ...) :
Status code was not 200. Returned status code:429
Look at more recent replies to more recent issue. "It works for us" so maybe you emitted more requests than Google likes. It is hard for us to debug what happens on your computer which is why we generally ask for minimally verifiable complete examples.
Hello, I am also getting the "Status code was not 200. Returned status code:429" error. Does anybody know of an alternative package?
basically this R package doesn't work anymore, the authors could at least update that fact!!
Hi, I am using the following code and facing the error.
gtrends("Biden",
geo = 'US',
time = "2023-02-15 2023-05-14", onlyInterest = TRUE )
Error in get_widget(comparison_item, category, gprop, hl, cookie_url, :
widget$status_code == 200 is not TRUE
Works here:
> library(gtrendsR)
> res <- gtrends("Biden", geo = 'US', time = "2023-02-15 2023-05-14", onlyInterest = TRUE )
> str(res)
List of 1
$ interest_over_time:'data.frame': 89 obs. of 7 variables:
..$ date : POSIXct[1:89], format: "2023-02-15" "2023-02-16" "2023-02-17" ...
..$ hits : int [1:89] 35 48 42 46 32 49 67 64 51 38 ...
..$ keyword : chr [1:89] "Biden" "Biden" "Biden" "Biden" ...
..$ geo : chr [1:89] "US" "US" "US" "US" ...
..$ time : chr [1:89] "2023-02-15 2023-05-14" "2023-02-15 2023-05-14" "2023-02-15 2023-05-14" "2023-02-15 2023-05-14" ...
..$ gprop : chr [1:89] "web" "web" "web" "web" ...
..$ category: int [1:89] 0 0 0 0 0 0 0 0 0 0 ...
- attr(*, "class")= chr [1:2] "gtrends" "list"
>
> plot(res)
> packageVersion("gtrendsR")
[1] ‘1.5.1.9000’
>
with
I'm having the same trouble.
I've updated R, R Studio, and all of my packages. Trying to replicate any of the simple queries above has resulted in "Error in interest_over_time(widget, comparison_item, tz) : Status code was not 200. Returned status code:429."
I've restarted several times and used a VPN. I've also used polite::bow, which has made the difference before, but nothing's working today.
Is it normal for there to be issues on the dev or Google side that just have to be cleared up or could it still be something on my end?
bow("https://trends.google.com/")
gtrends("Biden", geo = 'US', time = "2023-02-15 2023-05-14", onlyInterest = TRUE )
Seeing the issue myself again today. Worked fine before. Google trends website works fine.
Same problem: Error in interest_over_time(widget, comparison_item, tz) :Status code was not 200. Returned status code:429
Im still getting the same error message. This is the first time I use gtrends in about 2 years and this error is the only response i get.
gtrends( keyword = "elecciones", geo = "DO", gprop = "web", time = "today+5-y", category = 0, onlyInterest = TRUE )
Error in interest_over_time(widget, comparison_item, tz) : Status code was not 200. Returned status code:429
Works on my side:
[ins] r$> gtrends( keyword = "elecciones", geo = "DO", gprop = "web", time = "today+5-y", category = 0, onlyInterest = TRUE )
$interest_over_time
date hits keyword geo time gprop category
1 2019-07-07 <1 elecciones DO today+5-y web 0
2 2019-07-14 <1 elecciones DO today+5-y web 0
3 2019-07-21 <1 elecciones DO today+5-y web 0
4 2019-07-28 1 elecciones DO today+5-y web 0
5 2019-08-04 1 elecciones DO today+5-y web 0
6 2019-08-11 1 elecciones DO today+5-y web 0
7 2019-08-18 1 elecciones DO today+5-y web 0
8 2019-08-25 1 elecciones DO today+5-y web 0
9 2019-09-01 1 elecciones DO today+5-y web 0
10 2019-09-08 2 elecciones DO today+5-y web 0
11 2019-09-15 2 elecciones DO today+5-y web 0
12 2019-09-22 3 elecciones DO today+5-y web 0
13 2019-09-29 8 elecciones DO today+5-y web 0
14 2019-10-06 29 elecciones DO today+5-y web 0
15 2019-10-13 1 elecciones DO today+5-y web 0
16 2019-10-20 2 elecciones DO today+5-y web 0
17 2019-10-27 2 elecciones DO today+5-y web 0
18 2019-11-03 1 elecciones DO today+5-y web 0
19 2019-11-10 1 elecciones DO today+5-y web 0
20 2019-11-17 1 elecciones DO today+5-y web 0
21 2019-11-24 1 elecciones DO today+5-y web 0
22 2019-12-01 1 elecciones DO today+5-y web 0
23 2019-12-08 1 elecciones DO today+5-y web 0
24 2019-12-15 <1 elecciones DO today+5-y web 0
25 2019-12-22 <1 elecciones DO today+5-y web 0
26 2019-12-29 <1 elecciones DO today+5-y web 0
27 2020-01-05 1 elecciones DO today+5-y web 0
28 2020-01-12 1 elecciones DO today+5-y web 0
29 2020-01-19 1 elecciones DO today+5-y web 0
30 2020-01-26 1 elecciones DO today+5-y web 0
31 2020-02-02 2 elecciones DO today+5-y web 0
32 2020-02-09 4 elecciones DO today+5-y web 0
33 2020-02-16 13 elecciones DO today+5-y web 0
34 2020-02-23 2 elecciones DO today+5-y web 0
35 2020-03-01 3 elecciones DO today+5-y web 0
36 2020-03-08 3 elecciones DO today+5-y web 0
37 2020-03-15 39 elecciones DO today+5-y web 0
38 2020-03-22 2 elecciones DO today+5-y web 0
39 2020-03-29 1 elecciones DO today+5-y web 0
40 2020-04-05 1 elecciones DO today+5-y web 0
41 2020-04-12 2 elecciones DO today+5-y web 0
42 2020-04-19 1 elecciones DO today+5-y web 0
43 2020-04-26 1 elecciones DO today+5-y web 0
44 2020-05-03 1 elecciones DO today+5-y web 0
45 2020-05-10 1 elecciones DO today+5-y web 0
46 2020-05-17 1 elecciones DO today+5-y web 0
47 2020-05-24 2 elecciones DO today+5-y web 0
48 2020-05-31 2 elecciones DO today+5-y web 0
49 2020-06-07 1 elecciones DO today+5-y web 0
50 2020-06-14 2 elecciones DO today+5-y web 0
51 2020-06-21 3 elecciones DO today+5-y web 0
52 2020-06-28 5 elecciones DO today+5-y web 0
53 2020-07-05 49 elecciones DO today+5-y web 0
54 2020-07-12 2 elecciones DO today+5-y web 0
55 2020-07-19 1 elecciones DO today+5-y web 0
56 2020-07-26 1 elecciones DO today+5-y web 0
57 2020-08-02 <1 elecciones DO today+5-y web 0
58 2020-08-09 1 elecciones DO today+5-y web 0
59 2020-08-16 1 elecciones DO today+5-y web 0
60 2020-08-23 <1 elecciones DO today+5-y web 0
61 2020-08-30 <1 elecciones DO today+5-y web 0
62 2020-09-06 <1 elecciones DO today+5-y web 0
63 2020-09-13 <1 elecciones DO today+5-y web 0
64 2020-09-20 <1 elecciones DO today+5-y web 0
65 2020-09-27 1 elecciones DO today+5-y web 0
66 2020-10-04 <1 elecciones DO today+5-y web 0
67 2020-10-11 <1 elecciones DO today+5-y web 0
68 2020-10-18 1 elecciones DO today+5-y web 0
69 2020-10-25 1 elecciones DO today+5-y web 0
70 2020-11-01 100 elecciones DO today+5-y web 0
71 2020-11-08 7 elecciones DO today+5-y web 0
72 2020-11-15 2 elecciones DO today+5-y web 0
73 2020-11-22 1 elecciones DO today+5-y web 0
74 2020-11-29 1 elecciones DO today+5-y web 0
75 2020-12-06 1 elecciones DO today+5-y web 0
76 2020-12-13 <1 elecciones DO today+5-y web 0
77 2020-12-20 <1 elecciones DO today+5-y web 0
78 2020-12-27 <1 elecciones DO today+5-y web 0
79 2021-01-03 1 elecciones DO today+5-y web 0
80 2021-01-10 <1 elecciones DO today+5-y web 0
81 2021-01-17 <1 elecciones DO today+5-y web 0
82 2021-01-24 <1 elecciones DO today+5-y web 0
83 2021-01-31 1 elecciones DO today+5-y web 0
84 2021-02-07 1 elecciones DO today+5-y web 0
85 2021-02-14 1 elecciones DO today+5-y web 0
86 2021-02-21 <1 elecciones DO today+5-y web 0
87 2021-02-28 1 elecciones DO today+5-y web 0
88 2021-03-07 1 elecciones DO today+5-y web 0
89 2021-03-14 1 elecciones DO today+5-y web 0
90 2021-03-21 <1 elecciones DO today+5-y web 0
91 2021-03-28 <1 elecciones DO today+5-y web 0
92 2021-04-04 <1 elecciones DO today+5-y web 0
93 2021-04-11 1 elecciones DO today+5-y web 0
94 2021-04-18 <1 elecciones DO today+5-y web 0
95 2021-04-25 <1 elecciones DO today+5-y web 0
96 2021-05-02 1 elecciones DO today+5-y web 0
97 2021-05-09 <1 elecciones DO today+5-y web 0
98 2021-05-16 <1 elecciones DO today+5-y web 0
99 2021-05-23 <1 elecciones DO today+5-y web 0
100 2021-05-30 <1 elecciones DO today+5-y web 0
101 2021-06-06 3 elecciones DO today+5-y web 0
102 2021-06-13 1 elecciones DO today+5-y web 0
103 2021-06-20 <1 elecciones DO today+5-y web 0
104 2021-06-27 <1 elecciones DO today+5-y web 0
105 2021-07-04 <1 elecciones DO today+5-y web 0
106 2021-07-11 <1 elecciones DO today+5-y web 0
107 2021-07-18 <1 elecciones DO today+5-y web 0
108 2021-07-25 <1 elecciones DO today+5-y web 0
109 2021-08-01 <1 elecciones DO today+5-y web 0
110 2021-08-08 <1 elecciones DO today+5-y web 0
111 2021-08-15 <1 elecciones DO today+5-y web 0
112 2021-08-22 <1 elecciones DO today+5-y web 0
113 2021-08-29 <1 elecciones DO today+5-y web 0
114 2021-09-05 <1 elecciones DO today+5-y web 0
115 2021-09-12 <1 elecciones DO today+5-y web 0
116 2021-09-19 <1 elecciones DO today+5-y web 0
117 2021-09-26 <1 elecciones DO today+5-y web 0
118 2021-10-03 <1 elecciones DO today+5-y web 0
119 2021-10-10 1 elecciones DO today+5-y web 0
120 2021-10-17 <1 elecciones DO today+5-y web 0
121 2021-10-24 <1 elecciones DO today+5-y web 0
122 2021-10-31 <1 elecciones DO today+5-y web 0
123 2021-11-07 1 elecciones DO today+5-y web 0
124 2021-11-14 <1 elecciones DO today+5-y web 0
125 2021-11-21 1 elecciones DO today+5-y web 0
126 2021-11-28 1 elecciones DO today+5-y web 0
127 2021-12-05 <1 elecciones DO today+5-y web 0
128 2021-12-12 <1 elecciones DO today+5-y web 0
129 2021-12-19 <1 elecciones DO today+5-y web 0
130 2021-12-26 <1 elecciones DO today+5-y web 0
131 2022-01-02 <1 elecciones DO today+5-y web 0
132 2022-01-09 <1 elecciones DO today+5-y web 0
133 2022-01-16 <1 elecciones DO today+5-y web 0
134 2022-01-23 <1 elecciones DO today+5-y web 0
135 2022-01-30 <1 elecciones DO today+5-y web 0
136 2022-02-06 1 elecciones DO today+5-y web 0
137 2022-02-13 1 elecciones DO today+5-y web 0
138 2022-02-20 <1 elecciones DO today+5-y web 0
139 2022-02-27 <1 elecciones DO today+5-y web 0
140 2022-03-06 <1 elecciones DO today+5-y web 0
141 2022-03-13 1 elecciones DO today+5-y web 0
142 2022-03-20 1 elecciones DO today+5-y web 0
143 2022-03-27 <1 elecciones DO today+5-y web 0
144 2022-04-03 1 elecciones DO today+5-y web 0
145 2022-04-10 <1 elecciones DO today+5-y web 0
146 2022-04-17 <1 elecciones DO today+5-y web 0
147 2022-04-24 1 elecciones DO today+5-y web 0
148 2022-05-01 <1 elecciones DO today+5-y web 0
149 2022-05-08 <1 elecciones DO today+5-y web 0
150 2022-05-15 <1 elecciones DO today+5-y web 0
151 2022-05-22 <1 elecciones DO today+5-y web 0
152 2022-05-29 1 elecciones DO today+5-y web 0
153 2022-06-05 <1 elecciones DO today+5-y web 0
154 2022-06-12 1 elecciones DO today+5-y web 0
155 2022-06-19 1 elecciones DO today+5-y web 0
156 2022-06-26 <1 elecciones DO today+5-y web 0
157 2022-07-03 <1 elecciones DO today+5-y web 0
158 2022-07-10 <1 elecciones DO today+5-y web 0
159 2022-07-17 <1 elecciones DO today+5-y web 0
160 2022-07-24 <1 elecciones DO today+5-y web 0
161 2022-07-31 <1 elecciones DO today+5-y web 0
162 2022-08-07 <1 elecciones DO today+5-y web 0
163 2022-08-14 <1 elecciones DO today+5-y web 0
164 2022-08-21 <1 elecciones DO today+5-y web 0
165 2022-08-28 <1 elecciones DO today+5-y web 0
166 2022-09-04 <1 elecciones DO today+5-y web 0
167 2022-09-11 <1 elecciones DO today+5-y web 0
168 2022-09-18 <1 elecciones DO today+5-y web 0
169 2022-09-25 <1 elecciones DO today+5-y web 0
170 2022-10-02 2 elecciones DO today+5-y web 0
171 2022-10-09 <1 elecciones DO today+5-y web 0
172 2022-10-16 2 elecciones DO today+5-y web 0
173 2022-10-23 <1 elecciones DO today+5-y web 0
174 2022-10-30 2 elecciones DO today+5-y web 0
175 2022-11-06 3 elecciones DO today+5-y web 0
176 2022-11-13 1 elecciones DO today+5-y web 0
177 2022-11-20 1 elecciones DO today+5-y web 0
178 2022-11-27 <1 elecciones DO today+5-y web 0
179 2022-12-04 <1 elecciones DO today+5-y web 0
180 2022-12-11 <1 elecciones DO today+5-y web 0
181 2022-12-18 <1 elecciones DO today+5-y web 0
182 2022-12-25 <1 elecciones DO today+5-y web 0
183 2023-01-01 <1 elecciones DO today+5-y web 0
184 2023-01-08 <1 elecciones DO today+5-y web 0
185 2023-01-15 <1 elecciones DO today+5-y web 0
186 2023-01-22 <1 elecciones DO today+5-y web 0
187 2023-01-29 <1 elecciones DO today+5-y web 0
188 2023-02-05 <1 elecciones DO today+5-y web 0
189 2023-02-12 <1 elecciones DO today+5-y web 0
190 2023-02-19 <1 elecciones DO today+5-y web 0
191 2023-02-26 <1 elecciones DO today+5-y web 0
192 2023-03-05 <1 elecciones DO today+5-y web 0
193 2023-03-12 <1 elecciones DO today+5-y web 0
194 2023-03-19 <1 elecciones DO today+5-y web 0
195 2023-03-26 <1 elecciones DO today+5-y web 0
196 2023-04-02 <1 elecciones DO today+5-y web 0
197 2023-04-09 <1 elecciones DO today+5-y web 0
198 2023-04-16 1 elecciones DO today+5-y web 0
199 2023-04-23 1 elecciones DO today+5-y web 0
200 2023-04-30 1 elecciones DO today+5-y web 0
201 2023-05-07 1 elecciones DO today+5-y web 0
202 2023-05-14 1 elecciones DO today+5-y web 0
203 2023-05-21 <1 elecciones DO today+5-y web 0
204 2023-05-28 1 elecciones DO today+5-y web 0
205 2023-06-04 1 elecciones DO today+5-y web 0
206 2023-06-11 <1 elecciones DO today+5-y web 0
207 2023-06-18 <1 elecciones DO today+5-y web 0
208 2023-06-25 1 elecciones DO today+5-y web 0
209 2023-07-02 1 elecciones DO today+5-y web 0
210 2023-07-09 <1 elecciones DO today+5-y web 0
211 2023-07-16 1 elecciones DO today+5-y web 0
212 2023-07-23 2 elecciones DO today+5-y web 0
213 2023-07-30 <1 elecciones DO today+5-y web 0
214 2023-08-06 1 elecciones DO today+5-y web 0
215 2023-08-13 1 elecciones DO today+5-y web 0
216 2023-08-20 1 elecciones DO today+5-y web 0
217 2023-08-27 <1 elecciones DO today+5-y web 0
218 2023-09-03 <1 elecciones DO today+5-y web 0
219 2023-09-10 <1 elecciones DO today+5-y web 0
220 2023-09-17 1 elecciones DO today+5-y web 0
221 2023-09-24 1 elecciones DO today+5-y web 0
222 2023-10-01 3 elecciones DO today+5-y web 0
223 2023-10-08 1 elecciones DO today+5-y web 0
224 2023-10-15 1 elecciones DO today+5-y web 0
225 2023-10-22 2 elecciones DO today+5-y web 0
226 2023-10-29 1 elecciones DO today+5-y web 0
227 2023-11-05 1 elecciones DO today+5-y web 0
228 2023-11-12 1 elecciones DO today+5-y web 0
229 2023-11-19 2 elecciones DO today+5-y web 0
230 2023-11-26 1 elecciones DO today+5-y web 0
231 2023-12-03 1 elecciones DO today+5-y web 0
232 2023-12-10 <1 elecciones DO today+5-y web 0
233 2023-12-17 <1 elecciones DO today+5-y web 0
234 2023-12-24 <1 elecciones DO today+5-y web 0
235 2023-12-31 1 elecciones DO today+5-y web 0
236 2024-01-07 1 elecciones DO today+5-y web 0
237 2024-01-14 1 elecciones DO today+5-y web 0
238 2024-01-21 1 elecciones DO today+5-y web 0
239 2024-01-28 2 elecciones DO today+5-y web 0
240 2024-02-04 4 elecciones DO today+5-y web 0
241 2024-02-11 4 elecciones DO today+5-y web 0
242 2024-02-18 31 elecciones DO today+5-y web 0
243 2024-02-25 2 elecciones DO today+5-y web 0
244 2024-03-03 1 elecciones DO today+5-y web 0
245 2024-03-10 1 elecciones DO today+5-y web 0
246 2024-03-17 1 elecciones DO today+5-y web 0
247 2024-03-24 1 elecciones DO today+5-y web 0
248 2024-03-31 1 elecciones DO today+5-y web 0
249 2024-04-07 1 elecciones DO today+5-y web 0
250 2024-04-14 1 elecciones DO today+5-y web 0
251 2024-04-21 2 elecciones DO today+5-y web 0
252 2024-04-28 2 elecciones DO today+5-y web 0
253 2024-05-05 2 elecciones DO today+5-y web 0
254 2024-05-12 4 elecciones DO today+5-y web 0
255 2024-05-19 32 elecciones DO today+5-y web 0
256 2024-05-26 2 elecciones DO today+5-y web 0
257 2024-06-02 2 elecciones DO today+5-y web 0
258 2024-06-09 1 elecciones DO today+5-y web 0
259 2024-06-16 1 elecciones DO today+5-y web 0
260 2024-06-23 1 elecciones DO today+5-y web 0
261 2024-06-30 1 elecciones DO today+5-y web 0
262 2024-07-07 1 elecciones DO today+5-y web 0
attr(,"class")
[1] "gtrends" "list"
[ins] r$>