fluent-bit icon indicating copy to clipboard operation
fluent-bit copied to clipboard

Skip_Empty_Lines does not work for CR LF.

Open avizen-j opened this issue 1 year ago • 1 comments

I have encountered similar issue mentioned here - https://github.com/fluent/fluent-bit/issues/4215

Skip_Empty_Lines works only for LF empty lines. So in case you are running container on windows and while testing you manually create "log", empty line will contain CR LF which is ignored by this setting.

LF case image

CR LF case image

Result image

Potential resolution I assume the issue is here: https://github.com/fluent/fluent-bit/blob/3ce15c6f68d40075acbaabb5941036f278091c4c/plugins/in_tail/tail_file.c#L487C1-L496C1

Most likely crlf identification should be updated as well as if statement on L490 should be something like that: if (skip_empty_lines && crlf) or simply if (crlf)

Current one does make sense as 2nd if will never be true:

if (len >= 2) {
 crlf = (data[len-1] == '\r');
 if (len == 1 && crlf) {
  data += 2;
  processed_bytes += 2;
  continue;
 }
}

Sample config

[INPUT]
    Name tail
    Skip_Empty_Lines On
    Path /app-logs/*

[OUTPUT]
    Name stdout

avizen-j avatar Dec 21 '23 10:12 avizen-j

Thank you for reporting.

I sent a patch https://github.com/fluent/fluent-bit/pull/8474 .

nokute78 avatar Feb 10 '24 03:02 nokute78