fluent-plugin-s3 icon indicating copy to clipboard operation
fluent-plugin-s3 copied to clipboard

in_s3: enable multiline parsing

Open valentinacala opened this issue 2 years ago • 0 comments

Is your feature request related to a problem? Please describe.

I noticed me and many users are wondering to be able to recognize logs that are on many lines.

Currently any plugin seemes to satisfy the thing, because:

  • the fluentd multiline parsers applys on in_tail input only
  • the fluent-plugin-concat plugin works only if the numeber of lines is well defined

Describe the solution you'd like

I propose to embed the multiline parser in the fluent-plugin-s3 as it works for in_tail input

Describe alternatives you've considered

I tried the flluent-plugin-concat plugin, but it ignore the multiline_start_regexp if it is not specified a fixed number of lines

I tried also applying the multiline parser with S3 source, but I got error

Additional context

No response

valentinacala avatar Aug 08 '23 06:08 valentinacala