Suggested add: Ultimate htaccess Blacklist 2

[emanruse]

Hi,

Would you be willing to include the bots from this list?

https://perishablepress.com/ultimate-htaccess-blacklist/

I notice it contains user agents which your list does not or versions of UA's which look similar. (needs careful diff-ing)

Also it might be a good idea to have a single regex line combining all the bots, rather than multiple separate lines. AFAIK it is more resource efficient to process one big regex rather than many small ones. Please consider something similar to:

https://perishablepress.com/ultimate-htaccess-blacklist-2-compressed-version/

[emanruse]

AFAIK it is more resource efficient to process one big regex rather than many small ones.

To confirm my own words I made a test. Combining the bad referrers into 14 lengthy lines results in 5ms additional TTFB (limiting to 8192 characters as per Apache 2.2 docs). Without combining the TTFB is increased with 40ms. This is x8 difference.

[emanruse]

For regex compression I used:

# Bad user agents
RewriteCond %{HTTP_USER_AGENT} ^(.*)(\b(UA1|UA2|...|UAN)\b).* [NC,OR]
# Bad referrers
RewriteCond %{HTTP_REFERER} ^http(s)?://(www.)?.*(ref1|ref2|...|refNN).*$ [NC,OR]

But similarly to my comment in the other issue, I think this can probably be simplified because adding leading and trailing .* seems redundant. A regex like ^.*abc.*$ does the same as just abc.

[emanruse]

Sorry, I messed up the OP. This GitHub is driving me crazy.