Fail2Ban4Win icon indicating copy to clipboard operation
Fail2Ban4Win copied to clipboard

Should be able to monitor log files like the original Fail2Ban, not just Windows Events

Open JennaScvl opened this issue 1 year ago • 14 comments

I spent 3 hours trying to figure out why it wasn't loading when it turned out to be that it just can't monitor logs. For example I'd been trying to use this as one of my rules { "logName": "F:/xampp/apache/logs/error.log", "eventId": 0, "ipAddressPattern": "\[client (?<ipAddress>\d+\.\d+\.\d+\.\d+):\d+\]", "failurePattern": "AH00124" } and it just wouldn't load. The idea was to block an exploit attempt I keep seeing pop up in my Apache error.log Also { "logName": "F:/xampp/apache/logs/modsec_audit.log", "eventId": 0, "ipAddressPattern": ""client_ip":"(?<ipAddress>\d+\.\d+\.\d+\.\d+)"", "failurePattern": ""status":"218"" } This one is to monitor modsecurity's audit log to block certain common exploit attempts I've seen come in. But it just can't monitor log files like the original can, apparently.

JennaScvl avatar Jun 12 '24 18:06 JennaScvl

Just wanted to +1 this request. I have a similar challenge with IIS logs. I can have the IIS put its logs into the Event Viewer, but the IP address and the rest of the filtering criteria I'd need to account for are in different EventData Data fields, so I can't use Fail2Ban4Win for this particular challenge. It'd work if I could trigger off of additional fields in an EventLogSelector, or if I could work with the logfile directly, like @JennaScvl is requesting.

jmoeller-ua avatar Jul 10 '24 22:07 jmoeller-ua

Hi @jmoeller-ua, Feel free to export and upload an .evtx file for the IIS event(s), and I will try to take a look at them and see if they can be selected. Thanks.

Aldaviva avatar Jul 10 '24 23:07 Aldaviva

An example entry looks like this:

date 2024-07-10 time 23:18:54 s-sitename W3SVC1 s-computername <my_host> s-ip <my_ip> cs-method POST cs-uri-stem <my_url> cs-uri-query - s-port 443 cs-username - c-ip <remote_ip> cs-version - cs(User-Agent) Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/115.0.0.0+Safari/537.36+Edg/115.0.1901.203 cs(Cookie) - cs(Referer) - cs-host - sc-status 200 sc-substatus 0 sc-win32-status 0 sc-bytes 15322 cs-bytes 422 time-taken 351

The Event Viewer puts each element neatly and correctly into the EventData hashtable, but since I need to reference elements other than just c-ip to know if it's actually a block-worthy event (cs(User-Agent), cs-method, etc). If I could work with the entire message via regexp, just like a line in a logfile, I could be off and running.

Unhelpfully, the IIS logger puts in all events with the same event ID of 6200. But I guess it makes sense for a web log that it wouldn't be able to discern outcomes.

jmoeller-ua avatar Jul 10 '24 23:07 jmoeller-ua

That wasn't an .evtx file, but I figured out how to generate my own:

iis-access-log.evtx

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
    <System>
        <Provider Name="Microsoft-Windows-IIS-Logging" Guid="{7e8ad27f-b271-4ea2-a783-a47bde29143b}" />
        <EventID>6200</EventID>
        <Version>0</Version>
        <Level>4</Level>
        <Task>0</Task>
        <Opcode>0</Opcode>
        <Keywords>0x8000000000000000</Keywords>
        <TimeCreated SystemTime="2024-07-10T23:39:12.6648021Z" />
        <EventRecordID>1</EventRecordID>
        <Correlation />
        <Execution ProcessID="10412" ThreadID="10788" />
        <Channel>Microsoft-IIS-Logging/Logs</Channel>
        <Computer>Thor.aldaviva.com</Computer>
        <Security UserID="S-1-5-18" />
    </System>
    <EventData>
        <Data Name="EnabledFieldsFlags">2478079</Data>
        <Data Name="date">2024-07-10</Data>
        <Data Name="time">23:39:09</Data>
        <Data Name="c-ip">73.202.30.161</Data>
        <Data Name="cs-username">-</Data>
        <Data Name="s-sitename">W3SVC1</Data>
        <Data Name="s-computername">Thor</Data>
        <Data Name="s-ip">192.168.1.17</Data>
        <Data Name="cs-method">GET</Data>
        <Data Name="cs-uri-stem">/health-check.txt</Data>
        <Data Name="cs-uri-query">-</Data>
        <Data Name="sc-status">304</Data>
        <Data Name="sc-win32-status">0</Data>
        <Data Name="sc-bytes">0</Data>
        <Data Name="cs-bytes">766</Data>
        <Data Name="time-taken">103</Data>
        <Data Name="s-port">443</Data>
        <Data Name="csUser-Agent">Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/126.0.0.0+Safari/537.36</Data>
        <Data Name="csCookie">-</Data>
        <Data Name="csReferer">-</Data>
        <Data Name="cs-version">-</Data>
        <Data Name="cs-host">-</Data>
        <Data Name="sc-substatus">0</Data>
        <Data Name="CustomFields" />
    </EventData>
</Event>

This can be enabled by going to the Logging feature page for a site in IIS Manager, configuring it to use W3C format and log ETW events, and enabling and watching Event Viewer > Application and Service Logs > Microsoft > Windows > IIS-Logging > Logs.


Given this structure, it seems like allowing an additional optional configuration property to be specified in the EventLogSelector to act as a predicate in the XPath query generated by selectorToQuery would allow you to filter based on the user agent string, request method, response status code, and other data element values.

Aldaviva avatar Jul 11 '24 00:07 Aldaviva

That wasn't an .evtx file,

Indeed, I just wanted to strip out the identifying information. I had set it up just as you had mentioned.

Given this structure, it seems like allowing an additional optional configuration property to be specified in the EventLogSelector to act as a predicate in the XPath query generated by selectorToQuery would allow you to filter based on the user agent string, request method, response status code, and other data element values.

Yes, that's sort of what I was imagining, additional optional configuration items that I could refer to put a specific event in scope of Fail2Ban4Win. Would love to see it if you think it'd be a reasonable feature to implement.

jmoeller-ua avatar Jul 11 '24 00:07 jmoeller-ua

Right, those ETW instructions were for me and anyone else who looks at this issue in the future, because I had to figure it out and was worried I'd forget next time I have to do it (when updating unit tests, adding new dependent features, etc).

I believe adding the additional XPath predicate to the configuration is a reasonable feature to add. I will try to find the time soon to try it out and send you a development snapshot build to see if it fits your use case too before releasing it.

Aldaviva avatar Jul 11 '24 00:07 Aldaviva

@jmoeller-ua: give this a try and let me know how well it fits your use case

Developer snapshot executable

Fail2Ban4Win.zip (minimally tested so far)

Configuration

There is now a new optional property of EventLogSelector configuration objects with the key eventPredicate and the value of a string. The value is an XPath 1.0 predicate expression, which must be surrounded by square brackets and is evaluated by ETW against the top-level Event object of the ETW log record element.

Multiple predicates can be constructed with XPath and and or operators, or by concatenating multiple predicates like [predicate1][predicate2] instead of using the and operator. Note that functions like contains() and starts-with() are unfortunately not supported by Windows ETW.

Example

This selector should select only IIS access logs where the response status code is 403.

{
    "logName": "Microsoft-Windows-IIS-Logging/Logs",
    "source": "IIS-Logging",
    "eventId": 6200,
    "ipAddressEventDataName": "c-ip",
    "eventPredicate": "[EventData/Data[@Name='sc-status']=403]"
}

This will result in Fail2Ban4Win using an effective XPath expression of

*[System/EventID=6200][System/Provider/@Name=\"IIS-Logging\"][EventData/Data[@Name='sc-status']=403]

Here is another example that matches requests with both status code 304 and request method GET.

{
    "eventPredicate": "[EventData[Data[@Name='sc-status']='304'][Data[@Name='cs-method']='GET']]"
    /* other properties are the same */
}

Here is an example of an or expression to match two different status codes.

{
    "eventPredicate": "[EventData/Data[@Name='sc-status']='304' or EventData/Data[@Name='sc-status']='403']"
    /* other properties are the same */
}

Aldaviva avatar Jul 11 '24 02:07 Aldaviva

That would be fantastic. I'd be happy to try it out.


From: Ben Hutchison @.> Sent: Wednesday, July 10, 2024 5:42:11 PM To: Aldaviva/Fail2Ban4Win @.> Cc: Moeller, John D - (jmoeller) @.>; Mention @.> Subject: [EXT] Re: [Aldaviva/Fail2Ban4Win] Should be able to monitor log files like the original Fail2Ban, not just Windows Events (Issue #33)

External Email


Right, those ETW instructions were for me and anyone else who looks at this issue in the future, because I had to figure it out and was worried I'd forget next time I have to do it (when updating unit tests, adding new dependent features, etc).

I believe adding the additional XPath predicate to the configuration is a reasonable feature to add. I will try to find the time soon to try it out and send you a development snapshot build to see if it fits your use case too before releasing it.

— Reply to this email directly, view it on GitHubhttps://github.com/Aldaviva/Fail2Ban4Win/issues/33#issuecomment-2221774582, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APM4GNXCFXGYFIEAQ5THIBLZLXIGHAVCNFSM6AAAAABJG57YP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRRG43TINJYGI. You are receiving this because you were mentioned.Message ID: @.***>

jmoeller-ua avatar Jul 11 '24 06:07 jmoeller-ua

@jmoeller-ua: give this a try and let me know how well it fits your use case

Developer snapshot executable

Fail2Ban4Win.zip (minimally tested so far)

It works great, this is amazing. I can't believe how fast you did this!

jmoeller-ua avatar Jul 11 '24 22:07 jmoeller-ua

Thanks! Great to hear it's working for you.

This was a pretty straightforward change because it leveraged Windows' existing ETW XPath filtering, which this project was already using. By adding the one new optional, backwards-compatible configuration property, it augmented the existing Event ID and Source filtering that this project had already been set up to use. So it was a small change that did not reinvent the wheel.

Aldaviva avatar Jul 12 '24 04:07 Aldaviva

I'm splitting the eventPredicate feature that we've been discussing here into a different issue, #34, because it resulted in a different implementation than the topic of this issue, tailing text log files. Any further eventPredicate discussions can happen in #34, while this issue can remain for text log files.

Aldaviva avatar Jul 15 '24 04:07 Aldaviva

Hi there. I'm jumping pretty late in this, but I'd like to know if using the same format/example you provided, I could do the same for MailEnable (mail server for Windows). I'd like to monitor the various MailEnable logs (SMTP, POP, ...) and have the same functionality as with IIS shown above. Would that be a possibility?

Thanks for this wonderful tool! It really fills in a gap in security for anybody concerned about the crazy [internet] world we live in!

Cheers!

olikov avatar May 07 '25 19:05 olikov

Hi @olikov,

I'd have to see what the MailEnable log events look like. I personally use a different mail server so I'm not familiar with that one. Feel free to export some failing authentication event logs and upload them to a new GitHub issue if you want.

Aldaviva avatar May 08 '25 01:05 Aldaviva

Thanks a lot for the fast response!

I've created an issue with the details. Let me know if something is missing, or if you need more information.

On Wed, May 7, 2025 at 9:31 PM Ben Hutchison @.***> wrote:

Aldaviva left a comment (Aldaviva/Fail2Ban4Win#33) https://github.com/Aldaviva/Fail2Ban4Win/issues/33#issuecomment-2861151662

Hi @olikov https://github.com/olikov,

I'd have to see what the MailEnable log events look like. I personally use a different mail server so I'm not familiar with that one. Feel free to export some failing authentication event logs and upload them to a new GitHub issue if you want.

— Reply to this email directly, view it on GitHub https://github.com/Aldaviva/Fail2Ban4Win/issues/33#issuecomment-2861151662, or unsubscribe https://github.com/notifications/unsubscribe-auth/BAGJQ76YX5S2FEXKCFZSWAD25KXXLAVCNFSM6AAAAAB4UXEF3SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQNRRGE2TCNRWGI . You are receiving this because you were mentioned.Message ID: @.***>

olikov avatar May 08 '25 12:05 olikov