fluent-plugin-beats icon indicating copy to clipboard operation
fluent-plugin-beats copied to clipboard

How to get the data in the array by fluent?

Open rljie opened this issue 5 years ago • 0 comments

Framework filebeat --> kafka --> fluentd --> elasticsearch

Describe the problem filebeat comes with an array of tags, but fluentd doesn't filter out! The filter in logstash is

mutate {
     add_field =>  {
           "city" => "%{[tags][0]}"
           "ip" => "%{[tags][1]}"
     }
}

filebeat output configuration

output.kafka:
   hosts: ["120.0.0.1:9092"]
   topic: test
   required_acks: 1
tags: ["xinyi", "19.18.14.168"]
Fluentd Filter Configuration
<filter **>
   @type record_transformer
   enable_ruby true
   auto_typecast true
   <record>
      city %{[tags][0]}
      ip %{[tags][1]}
   </record>
</filter>

result The city and IP fields inserted into es are %{[tags][0]} and %{[tags][1} So, filtering didn't work, looking forward to answer, thanks

rljie avatar Dec 18 '19 08:12 rljie