fluent-bit
fluent-bit copied to clipboard
Missing record keys when using multiline java parser
Bug Report
Describe the bug Some output records do not contain keys like "filepath", "_p", "time", stream. Eg record [2] below
[0] tail.0: [[1708736407.169656515, {}], {"time"=>"2024-02-24T01:00:07.169656515Z", "stream"=>"stdout", "_p"=>"F", "log"=>"2022/11/29 05:41:10 ConsoleApplication1.MyCustomException: some message .... ---> System.Exception: Oh noes!
at ConsoleApplication1.SomeObject.OtherMethod() in C:\ConsoleApplication1\SomeObject.cs:line 24
at ConsoleApplication1.SomeObject..ctor() in C:\ConsoleApplication1\SomeObject.cs:line 14
--- End of inner exception stack trace ---
at ConsoleApplication1.SomeObject..ctor() in C:\ConsoleApplication1\SomeObject.cs:line 18
at ConsoleApplication1.Program.Main(String[] args) in C:\ConsoleApplication1\Program.cs:line 13", "filepath"=>"/var/log/containers/dotnet-qk7vm_default_dotnet-c8de97360d3de86d36bddfedf77392cd817b7c587d0bb16b1ece4c388ea7cef0.log"}]
[1] tail.0: [[1708736408.170722067, {}], {"time"=>"2024-02-24T01:00:08.170722067Z", "stream"=>"stdout", "_p"=>"F", "log"=>"2022/11/29 05:41:10 StackTrace: ' at System.Environment.GetStackTrace(Exception e)", "filepath"=>"/var/log/containers/dotnet-qk7vm_default_dotnet-c8de97360d3de86d36bddfedf77392cd817b7c587d0bb16b1ece4c388ea7cef0.log"}]
[2] tail.0: [[1708736407.169656515, {}], {"log"=>" at System.Environment.GetStackTrace(Exception e)
at System.Environment.get_StackTrace()
at Sample.Main()'"}]
[3] tail.0: [[1708736408.171001925, {}], {"time"=>"2024-02-24T01:00:08.171001925Z", "stream"=>"stdout", "_p"=>"F", "log"=>"2022/11/29 05:41:10 normal log", "filepath"=>"/var/log/containers/dotnet-qk7vm_default_dotnet-c8de97360d3de86d36bddfedf77392cd817b7c587d0bb16b1ece4c388ea7cef0.log"}]
To Reproduce
- Rubular link if applicable:
- Example log message if applicable:
- Steps to reproduce the problem:
2022/11/29 05:41:10 ConsoleApplication1.MyCustomException: some message .... ---> System.Exception: Oh noes!
at ConsoleApplication1.SomeObject.OtherMethod() in C:\ConsoleApplication1\SomeObject.cs:line 24
at ConsoleApplication1.SomeObject..ctor() in C:\ConsoleApplication1\SomeObject.cs:line 14
--- End of inner exception stack trace ---
at ConsoleApplication1.SomeObject..ctor() in C:\ConsoleApplication1\SomeObject.cs:line 18
at ConsoleApplication1.Program.Main(String[] args) in C:\ConsoleApplication1\Program.cs:line 13
2022/11/29 05:41:10 StackTrace: ' at System.Environment.GetStackTrace(Exception e)
at System.Environment.GetStackTrace(Exception e)
at System.Environment.get_StackTrace()
at Sample.Main()'
2022/11/29 05:41:10 normal log
save the logs above to a file and use as input for tail plugin with the config below.
Expected behavior Output record should have the necessary keys
Screenshots
Your Environment
- Version used: tested on linux 2.0.9 and 2.1.10
- Configuration:
[INPUT]
Name tail
Path /var/log/containers/*.log
Read_from_Head true
Multiline.Parser cri
Mem_Buf_Limit 10m
Buffer_Chunk_Size 1m
Buffer_Max_Size 1m
Rotate_Wait 20
Refresh_Interval 30
Path_Key filepath
Skip_Long_Lines On
[FILTER]
Name multiline
Match *
multiline.key_content log
multiline.parser java
- Environment name and version (e.g. Kubernetes? What version?):
- Server type and version:
- Operating System and version:
- Filters and plugins:
Additional context