theHarvester icon indicating copy to clipboard operation
theHarvester copied to clipboard

Published 20 hours ago • verified publisher icon laramies

[Enhancement] Support for Google Search API keys

Open advanced4 opened this issue 4 years ago • 2 comments

Feature Request Support using an API key for Google search, which would include Google dependent sources like Linkedin, Twitter, & Trello. Google & Linkedin are particularly great sources of information, but Google is very quick to rate limit requests.

Describe the feature request or bug or other Add a new source, "google_api" which supports using a Google Search API/CSE key

advanced4 avatar Sep 14 '20 16:09 advanced4

Google's rate limiting is a bit much which is why we plan on adding socks5 proxy capabilities eventually. Once that is added do you think pursuing the Google API key would still be worth it?

NotoriousRebel avatar Sep 14 '20 21:09 NotoriousRebel

I'm not sure - it would certainly help, but I tried earlier today with regular HTTP proxies and they are all already rate-limited by Google, so thus far proxies aren't helpful.

Google API includes (I think) 100 queries per day for free, which would go a long way. A similar project https://github.com/sundowndev/PhoneInfoga/blob/v1.11/lib/googlesearch.py#L99 implemented it this way. I've only glossed over theHarvester's code base, but hopefully this is a light lift to integrate.

advanced4 avatar Sep 14 '20 22:09 advanced4