kube-fluentd-operator icon indicating copy to clipboard operation
kube-fluentd-operator copied to clipboard

Exception thrown when `detect_exceptions` filter is not being used

Open cdancy opened this issue 3 years ago • 2 comments

Describe the bug

This is not so much a bug as it is an ask for how things are supposed to behave. We don't have detect_exceptions filter added but within our logs we have stacktraces dumped in between our json logs. When these logs eventually make their way over to NewRelic we see a lot of logs where stream: stderr but there is no message field present. There are also loads of these dumped seemingly at the same time.

Is this the expected behavior when the operator encounters a stacktrace and doesn't know what to do with it? Meaning it just creates a stream: stderr log with no message injected? That seems l suppose logical just want to confirm.

FWIW: we haven't enabled the detect_exceptions filter due to this currently open bug from over a year ago. We're not sure if this is still an issue with recent versions as we haven't tested but it's at least why we haven't tried it again.

Reproduction steps

Default behavior when `detect_exceptions` is not enabled but exceptions are encountered.

Expected behavior

Not sure actually which is why I'm here :)

Additional context

No response

cdancy avatar Sep 13 '22 20:09 cdancy

@javiercri does this make sense at all? Am I understanding things correctly? As you can see in NewRelic: it's just empty messages :) Uploading Screen Shot 2022-09-13 at 4.30.14 PM.png…

cdancy avatar Sep 13 '22 20:09 cdancy

Running in kubernetes, collecting log files in the container, application services with json logs and multi-line logs, how can I be compatible with this plug-in to support hot update

Only by adding the following configuration can json logs and multi-line logs be recognized normally

    <match raw.kubernetes.**>                   
      @id raw.kubernetes
      @type detect_exceptions                  
      remove_tag_prefix raw                     
      message log
      stream stream
      multiline_flush_interval 5
      max_bytes 500000
      max_lines 1000
    </match>

lvelvis avatar Jul 04 '23 12:07 lvelvis