For some reason, Fail2Ban, always blocks HTMLy login attempts

[n00b12345]

Whenever I try connecting remotely to my blog, (I use .htaccess to protect it) even after I enter the correct password, my attempt gets counted as a wrong attempt to log in.

This happens only with HTMLy, and not with other software I use on my server. Is there anything specific within HTMLy that does this?

[CodeAlDente]

It sounds more like a misconfiguration in your filters. Could you please post them?

[n00b12345]

@CodeAlDente thank you so much for helping. I really appreciate it.

Everything is set as default and the apache configuration in jail.local (which I use for configuration of fail2ban) is this:

[apache]

enabled  = true
port     = http,https
filter   = apache-auth
logpath  = /var/log/apache*/*error.log
maxretry = 2
bantime = -1
banaction = iptables-allports

[CodeAlDente]

Thank you.

I assume you are using the original filter apache-auth which can be found here: https://github.com/fail2ban/fail2ban/blob/0.10/config/filter.d/apache-auth.conf

Could you please provide the following information in order to check this:

• matching lines from your error.log (feel free to mask your ip-address)
• your fail2ban logfile

Please also post the output of /usr/bin/fail2ban-regex /var/log/apache*/*error.log /etc/fail2ban/filters.d/apache-auth.conf

According to http://stuffphilwrites.com/2013/03/permanently-ban-repeat-offenders-fail2ban/ , you may change your bantime to something like 31536000 (1 year) which is more practical than a "forever ban".

[n00b12345]

@CodeAlDente Thanks again. I hope this helps. Excuse the poor formatting.

This is the output of /var/log/fail2ban.log

2017-05-28 08:16:18,339 fail2ban.jail   [1324]: INFO    Jail 'apache' started
2017-05-28 09:43:00,072 fail2ban.actions[1324]: WARNING [apache] Ban x.x.x.x

This is the output of /var/log/apache2/error.log

[Sun May 28 09:42:54.091041 2017] [access_compat:error] [pid 1282] [client x.x.x.x:44014] AH01797: client denied by server configuration: /var/www/html/blog/
[Sun May 28 09:42:59.714891 2017] [access_compat:error] [pid 1281] [client x.x.x.x:20011] AH01797: client denied by server configuration: /var/www/html/blog/
[Sun May 28 09:42:59.728045 2017] [access_compat:error] [pid 1281] [client x.x.x.x:20011] AH01797: client denied by server configuration: /var/www/html/blog/index.php

/etc/fail2ban/filters.d/apache-auth.conf is untouched and is exactly the same as https://github.com/fail2ban/fail2ban/blob/0.10/config/filter.d/apache-auth.conf

[CodeAlDente]

Thank you.

Both logfiles look good as expected. The reason for the ban is that your request itself has been denied ("denied by server configuration") due to your server configuration, see https://github.com/fail2ban/fail2ban/blob/0.10/config/filter.d/apache-auth.conf#L17

Could you provide the content of your .htaccess please?

And you can remove everything below your line "This is the content of /usr/bin/fail2ban-regex" as we do not need the content but the actual output when you run the following command on your terminal:

/usr/bin/fail2ban-regex /var/log/apache*/*error.log /etc/fail2ban/filters.d/apache-auth.conf

[n00b12345]

@CodeAlDente This is what my my edited .htaccess looks like. These are the first few lines. The rest is the same as https://github.com/danpros/htmly/blob/master/.htaccess

I have allowed access for certain IPs on my local network.

AuthUserFile /home/user/.htpasswd
AuthType Basic
AuthName "My Secret Folder"
Require valid-user
Order Deny,Allow
Deny from all
Allow from 192.168.1.2
Allow from 192.168.1.3
Allow from 192.168.1.4
Allow from 192.168.1.5
Satisfy Any

And this is the output of /usr/bin/fail2ban-regex /var/log/apache*/*error.log /etc/fail2ban/filters.d/apache-auth.conf :

Running tests

Use failregex line : /etc/fail2ban/filters.d/apache-auth.conf Traceback` (most recent call last): File "/usr/bin/fail2ban-regex", line 430, in fail2banRegex.readRegex(cmd_regex, 'fail') or sys.exit(-1) File "/usr/bin/fail2ban-regex", line 227, in readRegex 'add%sRegex' % regextype.title())(regex.getFailRegex()) File "/usr/share/fail2ban/server/filter.py", line 95, in addFailRegex raise e server.failregex.RegexException: No 'host' group in '/etc/fail2ban/filters.d/apache-auth.conf'

---

I hope this helps. Thanks so much again for helping me.

[CodeAlDente]

Thank you.

I am so sorry. The command I asked you to run has a mistake. It should point to filter.d and not to filters.d as this is my custom configuration at my local machine. Sorry about that. Please run the command again on the "filter.d" folder as follows:

/usr/bin/fail2ban-regex /var/log/apache*/*error.log /etc/fail2ban/filter.d/apache-auth.conf

Make sure that you get banned by fail2ban before you run that command. That command tests the logfile for a possible match with the filter and will print out all requests that fail2ban would most likely ban. See also the logfile of fail2ban again to find out which ip address has been banned. Please confirm that it is none of the ip addresses that you have allowed in your .htaccess file. I had a similiar problem back in the days while I was using an IPv6 address instead of my usual one.

[n00b12345]

@CodeAlDente

This is what I get when I run /usr/bin/fail2ban-regex /var/log/apache*/*error.log /etc/fail2ban/filter.d/apache-auth.conf after I've been banned by fail2ban

---

Running tests

Use failregex file : /etc/fail2ban/filter.d/apache-auth.conf Use log file : /var/log/apache2/error.log

Results

Failregex: 3 total |- #) [# of hits] regular expression | 1) [2] ^[[^]]] [(:?error|\S+:\S+)]( [pid \d+(:\S+ \d+)?])? [client <HOST>(:\d{1,5})?] (AH01797: )?client denied by server configuration: (uri )?\S(, referer: \S+)?\s*$ | 2) [1] ^[[^]]] [(:?error|\S+:\S+)]( [pid \d+(:\S+ \d+)?])? [client <HOST>(:\d{1,5})?] (AH01617: )?user .? authentication failure for "\S*": Password Mismatch(, referer: \S+)?$ `-

Ignoreregex: 0 total

Date template hits: |- [# of hits] date format | [19] WEEKDAY MONTH Day Hour:Minute:Second[.subsecond] Year `-

Lines: 19 lines, 0 ignored, 3 matched, 16 missed |- Missed line(s): | [Tue May 30 06:25:14.438449 2017] [auth_digest:notice] [pid 1269] AH01757: generating secret for digest authentication ... | [Tue May 30 06:25:14.712787 2017] [ssl:warn] [pid 1269] AH01906: 127.0.1.1:443:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?) | [Tue May 30 06:25:14.713076 2017] [ssl:warn] [pid 1269] AH01909: 127.0.1.1:443:0 server certificate does NOT include an ID which matches the server name | [Tue May 30 06:25:14.718864 2017] [ssl:warn] [pid 1269] AH01909: o:443:0 server certificate does NOT include an ID which matches the server name | [Tue May 30 06:25:14.721724 2017] [mpm_prefork:notice] [pid 1269] AH00163: Apache/2.4.10 (Raspbian) OpenSSL/1.0.1t configured -- resuming normal operations | [Tue May 30 06:25:14.721863 2017] [core:notice] [pid 1269] AH00094: Command line: '/usr/sbin/apache2' | [Tue May 30 12:58:54.475858 2017] [mpm_prefork:notice] [pid 1269] AH00169: caught SIGTERM, shutting down | [Tue May 30 12:59:08.389809 2017] [ssl:warn] [pid 1189] AH01906: 127.0.1.1:443:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?) | [Tue May 30 12:59:08.391834 2017] [ssl:warn] [pid 1189] AH01909: 127.0.1.1:443:0 server certificate does NOT include an ID which matches the server name | [Tue May 30 12:59:08.399920 2017] [ssl:warn] [pid 1189] AH01909: o:443:0 server certificate does NOT include an ID which matches the server name | [Tue May 30 12:59:08.659817 2017] [auth_digest:notice] [pid 1265] AH01757: generating secret for digest authentication ... | [Tue May 30 12:59:09.265110 2017] [ssl:warn] [pid 1265] AH01906: 127.0.1.1:443:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?) | [Tue May 30 12:59:09.265491 2017] [ssl:warn] [pid 1265] AH01909: 127.0.1.1:443:0 server certificate does NOT include an ID which matches the server name | [Tue May 30 12:59:09.270381 2017] [ssl:warn] [pid 1265] AH01909: o:443:0 server certificate does NOT include an ID which matches the server name | [Tue May 30 12:59:09.300594 2017] [mpm_prefork:notice] [pid 1265] AH00163: Apache/2.4.10 (Raspbian) OpenSSL/1.0.1t configured -- resuming normal operations | [Tue May 30 12:59:09.300879 2017] [core:notice] [pid 1265] AH00094: Command line: '/usr/sbin/apache2'

---

And this is in the log file of fail2ban (/var/log/fail2ban.log)

2017-05-30 13:21:52,174 fail2ban.actions[1519]: WARNING [apache] Ban x.x.x.x

[CodeAlDente]

Thank you.

Did you check the ip-address that has been banned (Ban x.x.x.x)? Is it listed in your .htaccess file (one of those 192.168.x)?

[n00b12345]

Yes, the IP address was indeed banned and it didn't include any of those in the .htaccess exceptions.

The original issue persists, i.e. the IP address is banned even after entering the correct credentials for .htaccess

(Thank you again for responding. I really appreciate the help)

[andreas5232]

I came here by searching Google for a similar issue. My fail2ban using apache-auth filter was also hitting on those client denied by server configuration lines. In my case switching from old Order Deny,Allow style to the newer RequireAny apache 2.4 style auth configuration in the apache2 vhost settings fixed those unwanted-ban issues.

This is an example based on your previous apache config:

AuthUserFile /home/user/.htpasswd
AuthType Basic
AuthName "My Secret Folder"

<RequireAny>
   Require valid-user
   Require ip 192.168.1.2
   Require ip 192.168.1.3
   Require ip 192.168.1.4
   Require ip 192.168.1.5
</RequireAny>