fluent-bit icon indicating copy to clipboard operation
fluent-bit copied to clipboard

Remove_key in Record Modifier should be case sensitive

Open thwint opened this issue 1 year ago • 0 comments

Bug Report

Describe the bug We have some messages coming with a duplicate field. Once as hostName and once as hostname. The field hostName comes from the message field which is parsed from the given json. When defining a filter removing hostName, then both fields are removed.

To Reproduce Define a filter removing hostName:

      [FILTER]
          name            record_modifier
          match           isa.halfrunt.metric.**
          record          index isa-metric-10
          Remove_Key      timestamp
          Remove_Key      hostName
  • Example log message if applicable:
{"serviceName"=>"bangbang", "hostName"=>"bangbang-6546f7cdc8-zk8zd", "tags"=>[{"name"=>"domain", "value"=>"java.lang"}, {"name"=>"mbean_type", "value"=>"OperatingSystem"}, {"name"=>"attribute", "value"=>"OpenFileDescriptorCount"}], "values"=>[{"key"=>"value", "intValue"=>156}], "@timestamp"=>1734451912518, "environment"=>"snapshot", "source"=>"backend", "logger"=>"MetricLog", "hostname"=>"bangbang-6546f7cdc8-zk8zd", "service"=>"halfrunt", "index"=>"isa-metric-10"}
  • Steps to reproduce the problem:

Expected behavior Only the field hostName is removed from the record.

Screenshots

Your Environment

  • Version used: v3.2.1
  • Configuration:
  • Environment name and version (e.g. Kubernetes? What version?):
  • Server type and version:
  • Operating System and version:
  • Filters and plugins: record_modifier, parser, forward, output, lua

Additional context As a valid workaround we remove the hostName field using an ingest pipeline.

thwint avatar Dec 17 '24 16:12 thwint