fluent-bit
fluent-bit copied to clipboard
Remove_key in Record Modifier should be case sensitive
Bug Report
Describe the bug
We have some messages coming with a duplicate field. Once as hostName and once as hostname. The field hostName comes from the message field which is parsed from the given json. When defining a filter removing hostName, then both fields are removed.
To Reproduce
Define a filter removing hostName:
[FILTER]
name record_modifier
match isa.halfrunt.metric.**
record index isa-metric-10
Remove_Key timestamp
Remove_Key hostName
- Example log message if applicable:
{"serviceName"=>"bangbang", "hostName"=>"bangbang-6546f7cdc8-zk8zd", "tags"=>[{"name"=>"domain", "value"=>"java.lang"}, {"name"=>"mbean_type", "value"=>"OperatingSystem"}, {"name"=>"attribute", "value"=>"OpenFileDescriptorCount"}], "values"=>[{"key"=>"value", "intValue"=>156}], "@timestamp"=>1734451912518, "environment"=>"snapshot", "source"=>"backend", "logger"=>"MetricLog", "hostname"=>"bangbang-6546f7cdc8-zk8zd", "service"=>"halfrunt", "index"=>"isa-metric-10"}
- Steps to reproduce the problem:
Expected behavior
Only the field hostName is removed from the record.
Screenshots
Your Environment
- Version used: v3.2.1
- Configuration:
- Environment name and version (e.g. Kubernetes? What version?):
- Server type and version:
- Operating System and version:
- Filters and plugins: record_modifier, parser, forward, output, lua
Additional context
As a valid workaround we remove the hostName field using an ingest pipeline.