Generic Log Sources - Network protocols/services

[defensivedepth]

Zeek and Suricata generate overlapping datasets, specifically around protocol analysis. I would recommend that we look at creating some generic log sources focused on the overlapping protocol analysis fields. A good place to start would be smb.

[neu5ron]

a full taxonomy/schema 🙃 and here we are.. life has come full circle. in all seriousness, I think an ossem like taxonomy. Maybe just use it. regardless of the path, I agree on the vision to have one similar to how for process creation

[yugoslavskiy]

Hello @defensivedepth , @neu5ron!

My two cents:

(correct me if I am wrong) the main reason for having the logsources is the simplicity for rule developers via adding a layer of abstraction above similar log sources.

To make it work for us, the logsources should provide us with:

1. exact context of the field names (so we can correctly define the meaning of a detection logic on real data)
2. the ability to automatically map a rule to an index name while converting them

So, based on described, I believe that one of the ways could be:

1. Creation of the network category
2. Creation of the multiple services (or products):

• Protocol specific: smb, dns (already in place but on category level), http (already in place as proxy and also on category level), ftp etc. having separation by protocols will also allow us to develop Sigma backend for tcpdump or tshark. This approach is compliant with the listed requirements (exact context + automatic mapping) and goals (creating abstraction for rule developers)

• Network flows: they are mostly stored in a separate index and serve for multiple objectives (IR, Network Performance Monitoring etc). That's why I think it's better to keep them as a separate service/product.

• IPS/IDS specific: like Suricata, or in my case — Palo Alto Threat Log.

What do you guys think?

[neu5ron]

so should keep it at category level. thus can still have specifics of rules that may apply to a specific product/service. but keep the larger category for "all". like windows smb may have specifics, but want to include for all windows event IDs, yet not apply to zeek/suricata. so: category: smb product: windows

otherwise use: category: smb

i had already started SMB in ossem that has some windows events mapped and zeek mapped - @defensivedepth you have the suricata to zeek one to one for smb? if so, I can probably start this PoC pretty soon.. cc: @Cyb3rWard0g

[neu5ron]

i like "network flows" - just not the name :) unless somebody will be doing byte pattern matching.. just need a category for: 5 tuple src & dst bytes src & dst packets few other fields. basically the current "firewall" category - that works.. otherwise every vendor calls block/deny/reject/drop/break/stop.. differently.

[neu5ron]

and yeah again I don't like the network category, because in both practice and theory a lot of things can be network and endpoint rules - event 5145 & zeek/suricata smb, windows kerberos auth & zeek/suricata kerberos, windows dce rpc stuff & zeek/suricata, scheduled task or gpo modifications & zeek smb files/mappings, etc etc etc

[neu5ron]

@yugoslavskiy what do you think of my above comments?

[defensivedepth]

category sounds good to me!

[yugoslavskiy]

Hey @neu5ron ! Thanks for such a detailed answer!

As you said, there are many tricky things like collecting smb events from endpoints AND from network traffic analyzers. The borders are not clear.

I am not sure if the taxonomy/categorization is something we can all agree on. There is no right answer. That's why there are so many of them at the moment (:

Let's do something that will work for all of us, and for @defensivedepth specifically. I think that your proposal (network protocols on category level) will.

[thomaspatzke]

Hi all!

It makes definitive sense to create generic log sources for network protocols and map them via configurations to specific network or endpoint detection log sources like Zeek, Suricata or Windows logs.

Chosing the right log source naming is challenging here, there are arguments for both alternatives. Personally, I prefer the category: network naming for following reasons:

• The logs might be created by network devices and on endpoints, but the event is related to network activity. It's no problem to map network events to endpoint events at conversion time. This can be done by configuration.
• The decision if a rule is applicable generic or only to Windows shouldn't be done at rule writing time, but at conversion time (the new converter will also have a possibility to reject rules containing specific attributes by configuration - this was required for endpoint events anyways). Log sources might develop and support more attributes in future. Further log sources not considered by the rule author (or the participants of this thread) could support attributes. I think adding product: windows to a generic smb log source likely will be biased towards the environment of the rule author and cause lots of inconsistencies.
• In network terminology the term service is used for protocols, so it feels more natural to use it in log source definitions to refer to it.

Regarding the taxonomy for the generic log source I'm generally fine with OSSEM. It's a clean taxonomy and it's open (also in sense that it's not bound to any vendor).

[Cyb3rWard0g]

@neu5ron ! we should continue the conversations on this about helping integrating OSSEM for the generic log source (network wise at the moment). Let's talk more about it when we meet soon ;) . Looking forward to it!

Company agnostic open source projects working together would be amazing! 🙏 🍻