add parser for name=value format

[rgerhards]

[rgerhards]

We probably should support

name=value name="value" name='value'

In a single parser. I guess this does not mean the parser gets too broad.

[marnao]

Are you thinking this could be used to parse the ArcSight Common Event Format?

[rgerhards]

@marnao quick look at the manual suggest that this would most probably work at least for the extension part. Do you have any log records you could contribute to help developing this functionality? You email me them, for example. It would be great if I could include them into the log repository for research purposes (some early info on that here: http://blog.gerhards.net/2015/03/call-for-log-samples.html).

[marnao]

@rgerhards It would take a bit of effort for me to sanitize my logs here at work enough to share. If I can manage to generate any at home I will definitely share. I have been poking around on the internet trying to find some samples. I would also be happy to help test on a bigger data set once this is functional.

If I'm understanding correctly, we could capture the standard fields up front with normal parsing then use a recursive parser with this new functionality to cover the extensions? That would be an ideal solution in my mind, but not sure if it's possible.

There are quite a few CEF examples in here: https://www.f5.com/images/solution-center/arcsight-f5-asm-certified-cef-onfiguration-guide.pdf

The official CEF documentation (which I'm assuming you've seen) has a few: https://protect724.hp.com/servlet/JiveServlet/downloadBody/1072-102-6-4697/CommonEventFormat.pdf

[rgerhards]

Would you be willing to submit log samples if I can provide you with some tooling to do the sanitization? I am desperately looking for samples for a research project and it is very hard to do this if nobody contributes some ;) However, in order to do this tooling, I still need samples, albeit fewer.

I don't think we need the recursive parser. It's probably better to get away without it.

[rgerhards]

I've added a new tracker for the full Arcsight parser: https://github.com/rsyslog/liblognorm/issues/40

[aturetta]

I hade one already written as a rsyslog parser module, spotted this issue too late

[vdesabou]

@aturetta

I hade one already written as a rsyslog parser module, spotted this issue too late

Would it be something that you could share? I'm desperately looking for a way to parse CEF with rsyslog ! Thanks

[rgerhards]

The merged parser needs more work, and especially analysis of further samples. Thus I re-open the issue.

[davidelang]

even with a full cef parser, there is a lot of value in exposing a name-value parser as well.

a current example from the mailing list is trying to parse a uri string

name=value&name2=value2

the 'right' answer for this is %foo:tokenize:&:nvpair% to produce { "foo": ["name": "value", "name2", "value2"] }

[davidelang]

see https://github.com/rsyslog/liblognorm/issues/59 for info of a bug in the cef parser

[onlyjob]

name-value-list is a nice parser (thanks) but it would be great to extend it with customize-able separator in order to help parsing strings like

locale=AU;event_name=AB Testing;ab_stream=b;accesslevel=generic;

[davidelang]

It needs to have both the separator between pairs and within the pair to be configurable

name:value name:value name=value,name=value name:value|name:value

both values can be more than one character

name:=value, name:=value