elastic
decode_cef is unable to parse special characters in response field.
Description:
In the sample event shown below, the InfobloxDHCPOptions field includes special characters that decode_cef cannot parse, resulting in the field being dropped.
Sample event:
<134>1 2021-03-03T11:57:45Z - dataconnector - DHCP-LEASE-UPDATE - CEF:0|Infoblox|Data Connector|2.1.3|DHCP-LEASE-UPDATE|DHCP Lease Update|1|src=216.160.83.56 InfobloxClientID=01:00:0c:29:71:d8:95 InfobloxHostID=dhcp/host/94617 InfobloxFingerprintPr=false InfobloxRangeEnd= InfobloxRangeStart= smac=00:0c:29:71:d8:95 InfobloxIPSpaceName= InfobloxIPSpace=ipam/ip_space/a05fd0fd-b186-11ea-b69a-aa22288caa04 InfobloxSubnet= InfobloxFingerprint=osdfsd39429302sdfsdf2349 shost=wx-test-2. InfobloxLeaseUUID= InfobloxLifetime=3600 InfobloxLeaseOp=Update app=DHCP cat="DHCP Lease Update" InfobloxDUID= InfobloxDHCPOptions=code_12='ubuntu';code_53='003';code_55='001002006014017032034y003!()*w371374021';code_57='377377';code_61='001000014)t8e' InfobloxHost= dst=
Error Message:
Expectation:
The field should not be dropped and should instead be correctly parsed in the response returned by decode_cef
The Elastic CEF parser appears to be very unforgiving regarding unescaped characters or characters that have unnecessary escapes. It's quite frustrating because I was dealing with the same issue in terms of Infoblox Data Connector logs, and I had to write a Javascript to tweak the formatting before the decode_cef processor is used. Here's an example that covers the errors in my logs:
- script:
lang: javascript
source: >
function process(event) {
var msg = event.Get("event.original");
if (msg) {
msg = msg.replace(/(==?)(;|\" |\";])/g, function(match, c1, c2) {
if (c1 === '==') {
return '\\=\\=' + c2;
} else {
return '\\=' + c2;
}
});
msg = msg.replace(/(alpn|ipv4hint|ipv6hint|ech)=\\"([^"]*)\\"/g, function(match, key, value) {
return key + '="' + value + '"';
});
msg = msg.replace(/\\===? /g, '\\=\\= ');
msg = msg.replace(/destinationDnsDomain=(.*) spt/g, function(match) {
return match.replace(/\\./g, '.').replace(/\\ /g, ' ').replace(/\\/g, '\\\\');
});
msg = msg.replace(/msg=(\".*(?: \. ))/g, function(match, msgContents) {
return 'msg=' + msgContents.replace(/(\\)?(=)/g, function(match, escapeChar) {
if (escapeChar) {
return match;
}
return '\\=';
}).replace(/\\\\\./g, '.');
});
msg = msg.replace(/\\"/g, '"');
msg = msg.replace(/ dvchost$/, '');
event.Put("event.original", msg);
}
}
In your case, it seems like the CEF parser is complaining about an unescaped single quote. This isn't something I have encountered in my logs, so it wasn't in my script, but you are welcome to play around with it and see if it helps you.
Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices)
