logspout-logstash icon indicating copy to clipboard operation
logspout-logstash copied to clipboard

More issues with field names with dots in them

Open mpalmer opened this issue 9 years ago • 2 comments

Certain ill-mannered programs (I'm looking at you, docker-registry) emit JSON-formatted log entries, but use dot-delimited nesting in the key names, which breaks with ES 2.x. Since the JSON emitted by these programs isn't necessarily intended to be consumed by logstash, it's not necessarily their fault that they're producing logstash-incompatible JSON, but is instead logspout-logstash's responsibility to transmute the error-producing JSON into something that logstash will understand.

I can think of two options:

  1. Replace all dots in keys with underscores (quick, dirty, and within my extremely limited Go skills to implement); or
  2. Translate dot-encoded keys into a nested structure (far superior, but there's no way known I can implement that all by myself).

Would a PR to implement option 1 be merged, or should I wait for someone with better Go chops to implement 2?

mpalmer avatar May 20 '16 00:05 mpalmer

Hey, sorry for not responding sooner. Is this still an issue for you? Which solution would you prefer as a fix in that case?

maxekman avatar Aug 29 '16 17:08 maxekman

Yes, it's still an issue. Option 2 would be a better outcome, by far, because it maintains structure, but someone else would have to implement it because I've got near-zero Go skills. I could do a PR for option 1, though, if nobody else has the time to code something better.

mpalmer avatar Aug 30 '16 02:08 mpalmer