monperrus
Feature-Request: Difference between good and bad bots?
There should be two lists of good and bad bots. Example: Google and Bing are probably legitimate for most users.
Please indicate when it was added to the list to better find new entries. ;)
This is meant to be a list of good bots. Bad bots don't give user-agents or simply lie, and have to be detected with some more advanced techniques.
This is meant to be a list of good bots. Bad bots don't give user-agents or simply lie, and have to be detected with some more advanced techniques.
This is not factual. Bad bots do give UAs more times than not. Most webmasters with WAF's in place can confirm this. I've just began to look through the current list in place, and therefore cannot yet give definitive examples within it, although many bots listed are either unrecognizable or unwanted, by myself and, I suspect but cannot say with certainty by other site owners.
I will, as time permits, help to clarify my statement and point to specific examples as I find the work you do very valuable and of service to the community.
Thank-you!
I understand your point. Then, we would need a clear procedure to assess and maybe measure the goodness of bots.