GoogleScraper icon indicating copy to clipboard operation
GoogleScraper copied to clipboard

Suggestion towards overriding "Malicious Request Detected"

Open ethanchewy opened this issue 7 years ago • 0 comments

I've tested this for some weeks and just t wanted to give some results.

Overall, it was pretty good at first, but Google soon capped it to around 80 search results per day.

I see the proxy built in function. Is there a better way to overcome this?

Perhaps using something like: https://github.com/taspinar/twitterscraper/blob/master/twitterscraper/query.py#L8 and using a fake user_agent?

Again, huge fan of the project but wish someone would update it to avoid all the inconveniences when it come with grabbing more than a 1,000 links per day.

ethanchewy avatar Jan 12 '18 22:01 ethanchewy