fluent-bit icon indicating copy to clipboard operation
fluent-bit copied to clipboard

in_tail: skip_empty_line supports CRLF line

Open nokute78 opened this issue 1 year ago • 0 comments

Fixes #8313

This patch is to ignore \r\n lines when skip_empty_line is true.


Enter [N/A] in the box, if an item is not applicable to your change.

Testing Before we can approve your change; please submit the following in a comment:

  • [X] Example configuration file for the change
  • [X] Debug log output from testing the change
  • [X] Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • [N/A] Run local packaging test showing all targets (including any new ones) build.
  • [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • [N/A] Documentation required for this feature

Backporting

  • [ ] Backport to latest stable release.

Configuration

[INPUT]
    NAME   tail
    read_from_head on
    skip_empty_lines on
    path crlf.txt

[OUTPUT]
    Name   stdout

Debug/Valgrind output

Create crlf.txt.

printf "aa\r\n" >> crlf.txt
printf "\r\n" >> crlf.txt
printf "bb\r\n" >> crlf.txt
printf "\r\n" >> crlf.txt
printf "cc\r\n" >> crlf.txt

output:

$ valgrind --leak-check=full bin/fluent-bit -c a.conf 
==55467== Memcheck, a memory error detector
==55467== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==55467== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==55467== Command: bin/fluent-bit -c a.conf
==55467== 
Fluent Bit v3.0.0
* Copyright (C) 2015-2024 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

____________________
< Fluent Bit v2.2.2 >

[2024/02/10 12:17:44] [ info] [fluent bit] version=3.0.0, commit=978b280aba, pid=55467
[2024/02/10 12:17:44] [ info] [storage] ver=1.2.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/02/10 12:17:44] [ info] [cmetrics] version=0.6.6
[2024/02/10 12:17:44] [ info] [ctraces ] version=0.4.0
[2024/02/10 12:17:44] [ info] [input:tail:tail.0] initializing
[2024/02/10 12:17:44] [ info] [input:tail:tail.0] storage_strategy='memory' (memory only)
[2024/02/10 12:17:45] [ info] [output:stdout:stdout.0] worker #0 started
[2024/02/10 12:17:45] [ info] [sp] stream processor started
[2024/02/10 12:17:45] [ info] [input:tail:tail.0] inotify_fs_add(): inode=4291642 watch_fd=1 name=crlf.txt
[0] tail.0: [[1707535065.098682311, {}], {"log"=>"aa"}]
[1] tail.0: [[1707535065.130616451, {}], {"log"=>"bb"}]
[2] tail.0: [[1707535065.130788052, {}], {"log"=>"cc"}]
^C[2024/02/10 12:17:46] [engine] caught signal (SIGINT)
[2024/02/10 12:17:46] [ warn] [engine] service will shutdown in max 5 seconds
[2024/02/10 12:17:46] [ info] [input] pausing tail.0
[2024/02/10 12:17:47] [ info] [engine] service has stopped (0 pending tasks)
[2024/02/10 12:17:47] [ info] [input] pausing tail.0
[2024/02/10 12:17:47] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2024/02/10 12:17:47] [ info] [output:stdout:stdout.0] thread worker #0 stopped
[2024/02/10 12:17:47] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=4291642 watch_fd=1
==55467== 
==55467== HEAP SUMMARY:
==55467==     in use at exit: 0 bytes in 0 blocks
==55467==   total heap usage: 1,783 allocs, 1,783 frees, 783,250 bytes allocated
==55467== 
==55467== All heap blocks were freed -- no leaks are possible
==55467== 
==55467== For lists of detected and suppressed errors, rerun with: -s
==55467== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

nokute78 avatar Feb 10 '24 03:02 nokute78