fluent-bit-docs icon indicating copy to clipboard operation
fluent-bit-docs copied to clipboard

Multiline missing actual parse example

Open squalou opened this issue 3 years ago • 0 comments

Hi,

I'm struggling with "multiline" and I think the documentation is missing one example whare

  • lines are indeed "joinded"
  • THEN parsed to fields.

So far, following this documentation :

https://docs.fluentbit.io/manual/pipeline/filters/multiline-stacktrace

It's easy enough to get for instance a full stack trace with its line breaks inside a "log" field.

But then what ?

No idea how to parse this log. Spoiler alert : chaining another "filter parser" does not work, only the first line is parsed.

Real usecase would seem to be :

step one :

[1] tail.0: [1626736433.143570538, {"log"=>"Dec 14 06:41:08 Exception in thread "main" java.lang.RuntimeException: Something has gone wrong, aborting!
    at com.myproject.module.MyProject.badMethod(MyProject.java:22)
    at com.myproject.module.MyProject.oneMoreMethod(MyProject.java:18)

step two : obtain

{
  "time" : "Dec 14 06:41:08",
  "message" : "Exception in thread "main" java.lang.RuntimeException: Something has gone wrong, aborting!
    at com.myproject.module.MyProject.badMethod(MyProject.java:22)
    at com.myproject.module.MyProject.oneMoreMethod(MyProject.java:18)"
 }

Additional context : I cannot change my "INPUT" configuration, hence cannot rely on the Multiline on and Parser_Firstline multiline approach. I have to play with [FILTER] only. (AWS firelens context, not to enter in too many details)

squalou avatar Mar 03 '22 08:03 squalou