nginx-ultimate-bad-bot-blocker
nginx-ultimate-bad-bot-blocker copied to clipboard
mitchellkrogza
[CONTRIBUTE] Fake GoogleBots
A new feature added to the blocker recently is a growing list of Fake GoogleBots.
This was started by gracious contributions by @Eckybrazzz thank you.
Today I wrote a script which automatically processes a special Nginx log every day and emails me a list of Fake GoogleBots. It involves a combination of a separate Nginx logfile and a simple bash script.
I wrote it in under an hour and have got 5 fake GoogleBots in less than 2 hours from all my sites.
Tomorrow I will tweak and enhance the script to look for fake Bing, Ask and other popular bots too.
If anyone is willing to run this new script and the required mods to Nginx for the unique log file needed for this. Let me know as we will then add Fake GoogleBots at an alarming rate if more of the community is willing to jump in on this.
Just reply here and once I have this finalised I will publish the details
