Experimental OT protocols modules, new IDS/IPS feature, configurable logging, fixes for better thread safety

[wildstray]

We choose your script to make a PoC of parametric IDS/IPS for OT. There are five new modules: Modbus TCP, MQTT (sorry, I know, I overwrite the pre existing MQTT module), Siemens Logo! configuration and programming protocol, Siemens S7 (magic = 0x32) and a "default" module for "anything else" (to create flexible filters based on payload content ala Snort/Suricata). These modules can generate audit trails from payload (with variable level of verbosity), We also added to main script:

• python logging (also with logging.config support)
• review of modules execute() calling for better thread safety
• configurable filters with alert/pass/drop/reject actions

And a README.md enriched with explanations of the new features, options, examples, limitations. If you want, you might create an ot_experimental branch and accept the PR on it.

[ptitdoc]

Hello,

I'm checking your PR as we both have a big PR (https://github.com/ickerwx/tcpproxy/pull/21) with similar features (except for plugins).

Basically, I also implemented logging, and rules mechanism (however not through flat files and no options for bus protocols).

I also changed the plugins mechanism in order to allow peeking or changing the connection (allowing to implement DROP as a plugin).

The difference is that I use logging and rules mechanism through redis (using json) and not necessarily from a CVS flat file but this is not incompatible (this allows having a client/server model and a deported GUI)

Can you give me more details about what you have done regarding:

• review of modules execute() calling for better thread safety

[ickerwx]

It looks like @ptitdoc has spent some time to merge this and his own changes into #27 and #28. I'll leave this open for now until I can say for certain that his merge effort and your work match.

In any case, thank you very much for spending your time on improving this tool.

[ptitdoc]

Hello, bringing back work in progress here as the other pull requests have been closed:

More work is needed to merge filters for OT protocols. Here is the current approach:

• Filters currently only allow dynamically selecting which module are chained depending on the connection characteristics.
• However, modules can now peek data or change the connection properties (wrapping, dropping ...). drop is typically implemented as a module. Similarly alert can be implemented by modifying the log module.
• Basic filters are currently implemented in tcpproxy directly for parameters that don't need further inspection (src/dst/ports...).
• Advanced filters that need some parsing are currently implemented by the function is_inhibited in modules. For example the basemodule implement this function to verify if there is a rule regarding the hostname.
• The current filters are not implemented using an IPTable approach. It just allow chaining modules dynamically depending on the connection, and this makes harder to implement a DROP all approach. However I can experiment if a similar result can be achieved using the --do-not-chain parameter which would only run the first Module matching our filters in order.
• Information exchange between chained modules can be done through the connection objects. They keep track of basic information such as ips/ports and actions that have been taken by plugins (using tags).
• Not specifying --target-ip allow using transparent proxying using linux nat table.
• Specifying rules to load is possible through --rules file:///path/to/my/file.json, --rules file:///path/to/my/file*.csv, --rules redis://localhost
• For csv files instead of knowing the fields in advance, we could load it from the csv header.

Work that still need to be done:

• Loading rules from csv flat files (the code has been copied but is not functionnal yet).
• Fixing OT filtering plugins (especially the logic to take actions based on rules).