fluent-plugin-beats
fluent-plugin-beats copied to clipboard
How to get the data in the array by fluent?
Framework filebeat --> kafka --> fluentd --> elasticsearch
Describe the problem filebeat comes with an array of tags, but fluentd doesn't filter out! The filter in logstash is
mutate {
add_field => {
"city" => "%{[tags][0]}"
"ip" => "%{[tags][1]}"
}
}
filebeat output configuration
output.kafka:
hosts: ["120.0.0.1:9092"]
topic: test
required_acks: 1
tags: ["xinyi", "19.18.14.168"]
Fluentd Filter Configuration
<filter **>
@type record_transformer
enable_ruby true
auto_typecast true
<record>
city %{[tags][0]}
ip %{[tags][1]}
</record>
</filter>
result The city and IP fields inserted into es are %{[tags][0]} and %{[tags][1} So, filtering didn't work, looking forward to answer, thanks