Filebeat/auditd - Enable Grok regex for "ENRICHED" log_format with auditd

[rperigo]

Describe the enhancement: Using filebeat 7.6.2, auditd module. Unsure if this qualifies as bug or enhancement.

On Enterprise Linux 7 based systems (all I have ATM, this should probably be tested elsewhere), when setting log_format = ENRICHED in the system auditd config, the filebeat auditd module will by default choke on the append point for the additional human readable values. It ends up reporting auditd.log.res (or event.outcome) as something like success'UID="root. This is because when enabling ENRICHED logging, auditd places an unprintable char after the single-quote, followed directly by the new values. This breaks whitespace-based splits and causes the above garbage in the output.

After some digging, I've gotten this working on my stack by adjusting the relevant Grok pattern as follows (line 14 of modules/auditd/log/ingest/pipeline.yml):

'%{AUDIT_PREFIX} %{AUDIT_KEY_VALUES:auditd.log.kv} msg=[''"](%{DATA:auditd.log.msg}\s+)?%{AUDIT_KEY_VALUES:auditd.log.sub_kv}[''"]\x1d?%{AUDIT_KEY_VALUES:auditd.log.sub_sub_kv}'

And adding a new kv processor for the additional key:value set:

 - kv:
     field: auditd.log.sub_sub_kv
     field_split: "\\s+"
     value_split: "="
     target_field: auditd.log
     ignore_missing: true

In the above grok, the \x1d maps to the ASCII char auditd injects into the log line.

Describe a specific use case for the enhancement or feature:

On (at least) Red Hat and CentOS systems running auditd, it can be useful to enable ENRICHED logging to place human-readable user and group names directly into the log stream. This however will break the auditd modules parsing of said log, so out of the box one must choose one or the other.

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[rperigo]

It dawns on me I could just try to open a feature branch and submit a PR with the code I've changed. Not sure what you folks have as far as gitflow procedures though :)

[botelastic[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[adriansr]

Reopening this as it's a necessary enhancement

[botelastic[bot]]

Hi! We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

[efd6]

See #37771.