output/reference: Include reference information in alert (if configured)

[jlucovsky]

Continuation of #11079

When configured, include the reference value in the alert. The configuration value is in the alert section: types.alert.reference. The default value is off/no. Set to yes to include the expanded reference from the rule in the alert record.

Link to redmine ticket: 4974

Describe changes:

• Add reference value to suricata.yaml.in (default no/off)
• Set flag in output logger if the config setting is on
• Format the reference as a sequence, e.g., references: [ "ref-1" [, "ref-2" [, ...]]]

Updates:

• Rebase (and s-v rebase)

Provide values to any of the below to override the defaults.

SV_BRANCH=https://github.com/OISF/suricata-verify/pull/1808

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 94.44444% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 83.75%. Comparing base (fcdd7f0) to head (f5cc6fa). Report is 12 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #11089       +/-   ##
===========================================
+ Coverage   64.19%   83.75%   +19.55%     
===========================================
  Files         847      922       +75     
  Lines      136684   250478   +113794     
===========================================
+ Hits        87750   209785   +122035     
+ Misses      48934    40693     -8241     

Flag	Coverage Δ
fuzzcorpus	64.21% <22.22%> (+0.01%)	:arrow_up:
livemode	19.38% <22.22%> (?)
suricata-verify	62.79% <94.44%> (?)
unittests	62.23% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

[jasonish]

Given a rule with reference:url,www.vpngate.net;, I get the following logged:

    "references": [
      "http://www.vpngate.net"
    ],

what are the other possibilities?

I guess I had expected something more like:

"references": [
    {"scheme": "url", "value": "www.vpngate.net"}
]

[suricata-qa]

Information: QA ran without warnings.

Pipeline 20653

[jlucovsky]

Given a rule with reference:url,www.vpngate.net;, I get the following logged:

    "references": [
      "http://www.vpngate.net"
    ],

what are the other possibilities?

I guess I had expected something more like:

"references": [
    {"scheme": "url", "value": "www.vpngate.net"}
]

The reference associated with the signature has already been transformed from the scheme/value layout using reference.config. Thus, reference values available when adding to the alert are of the form { "http://", "www.vpgate.net"} Other possibilities are defined by the reference.config file and how the rules are used.

A more complicated example is a rule with a reference: reference:md5,8ec85e487dff0a868e1eed78e0c43460 This would be transformed into { "http://www.threatexpert.com/report.aspx?md5=", "8ec85e487dff0a868e1eed78e0c43460" } and included in the alert as "references":[ "http://www.threatexpert.com/report.aspx?md5=8ec85e487dff0a868e1eed78e0c43460" ]

[jlucovsky]

Continued in #11284