Supporting Humio and multiple Suricata probes

Open Potrik98 opened this issue 6 years ago • 1 comments

Adds support for using Humio as an optional replacement of ElasticSearch

/rules: alerts activity, alerts trend and rules activity
/rules/rule/pk/: ip and time stats, advanced data
/suricata: rules activity and alerts activity (including sunburst graph)
When using Humio, the ElasticSearch status indicator and status page is replaced with a Humio status indicator and status page.
/rules/hunt: dashboard, signatures and alerts works the same, including filtering.

Humio can be selected as one of the possible backends in settings.py.

Show data from multiple Suricata probes consistently across different ui elements
Selecting (enable/disable) probes on the timeline graph. This automatically filters the rules table on /rules, and the rules table and sunburst graph on the /suricata page

When ordering or filtering a table, the table is updated without having to refresh the page
When interacting with the sunburst on /suricata or the timeline on /rules or /suricata, the other ui elements are updated accordingly.

Overload the default comparator in the category model resolving the issue of inconsistent results when sorting by category 227398f5eb7359ba8bcd118499524843bf63ba77
Undefined variable access in disablecategory command ba50ebfaf485d5ce2e07260a8fbd21a5ae0cebdc
Change from floating point to integer on the timeline y axis and update the graph correctly c9c6f0c764c54d4d9e5c75a15b5bb3ba0ba9ba2a
Hide the 'Fetching data' text paragraph after the timeline is rendered 4d39fa5282ff0f68c3008461c5b49cb93f3ad1b1
Fix filtering on category on the /suricata page when using ElasticSearch with a keyword setting other than 'raw' 12ba989637451f6df6bcf9a44abf265e4f923ea9

Add a removesuricata command to manage.py removing a suricata probe by name cf8de6fb9ba82d6b6ff9ab5600975f21bed3d62c
Lazy computation of the queryset in RuleHitsOrderingFilter in rest_api, to avoid uneccesarily evaluating the entire queryset for all the rules in the database. This significantly reduces the response time (from ~11s to ~1s on my machine). 93a48f07e059bb943dd531a3a61e74afcedc79c8 ddc1466070e2d66e65f172863df2e037a45dba5e