hub
hub copied to clipboard
http-bad-user-agent is not detecting Netcraft
I tested with a testfile with two lines
fd:0:0:40::144 - - [03/Nov/2022:15:32:32 +0100] "GET /network HTTP/1.1" 200 13667 "-" "Mozilla/5.0 (compatible; NetcraftSurveyAgent/1.0; [email protected])" "fd:0:0:40::144"
fd:0:0:40::144 - - [03/Nov/2022:15:32:32 +0100] "GET /network HTTP/1.1" 200 13667 "-" "Mozilla/5.0 (compatible; zgrab)" "fd:0:0:40::144"
Second line is detected, first one in not. As far as I see, Netcraft is part of bad user agents lists.