Optimization of bad user agents

[mitchellkrogza]

Topic continues from https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker/issues/50

[zen2]

Hello ! Thanks for your project, I'm using your lists but differently because I've to match IP on a specific HTTP header issued by a CDN. BTW you can be interested how you can filter IP like bots:

SetEnvIfExpr "-R '10.0.0.0/8' || -R '172.16.0.0/12' || -R '192.168.0.0/16'" privateip
SetEnvIfExpr "%{HTTP:X-Forwarded-For} -ipmatch '23.21.227.69'" badip

I use last one on a different HTTP header to be able to filter behind a proxy/CDN without lost CDN Server IP.

I got recently a vulnerability scan as part of an attack on one of my web server. And I got thousands of requests related to Acunetix vulneralibities scanner with specific pattern in user agent or referrer or url:

User Agent:

  "${@print(md5(acunetix_wvs_security_test))}\\"
  "';print(md5(acunetix_wvs_security_test));$a='"
  "${@print(md5(acunetix_wvs_security_test))}"
  ";print(md5(acunetix_wvs_security_test));"
  "\";print(md5(acunetix_wvs_security_test));$a=\""

Not trapped because there is no space around "acunetix" by: BrowserMatchNoCase "(?:\b)Acunetix(?:\b)" badbot but trapped with simple: BrowserMatchNoCase "Acunetix" batbot

Referrer:

"${@print(md5(acunetix_wvs_security_test))}"
"${@print(md5(acunetix_wvs_security_test))}\\"
"http://www.acunetix-referrer.com/javascript:domxssExecutionSink(0,\"'\\\"><xsstag>()refdxss\")"
";print(md5(acunetix_wvs_security_test));"
"';print(md5(acunetix_wvs_security_test));$a='"
"\";print(md5(acunetix_wvs_security_test));$a=\""

URL (part of):

acunetix_wvs_security_test
$acunetix
acunetix-wvs-test-for-some-inexistent-file

[zen2]

Like Acunetix ones, I got thousands of requests with these UA:

"Http://testasp.vulnweb.com/t/fit.txt"
"http://testasp.vulnweb.com/t/fit.txt%3F.21"
"http://testasp.vulnweb.com/t/xss.html?%00.21"
"testasp.vulnweb.com"

I got too this kind of pattern UA (several thousands UA observed):

"http://hitzzCd2a2BVB.bxss.me/"
"http://hitzzHGwiPsIB.bxss.me/"
"http://hitZzi9wNlN1c.bxss.me/"
"http://hitzZSZb33Qi8.bxss.me/"
"http://hitZzXGICR0bU.bxss.me/"

And finally another UA from another vulnerability scanner: "Nuclei - Open-source project (github.com/projectdiscovery/nuclei)"