fluent-plugin-kafka
fluent-plugin-kafka copied to clipboard
fluent
Kafka input and output plugin for Fluentd
I am using this configuration: ``` @type kafka brokers somebrokers:9092 default_topic 'sometopic' output_data_type json max_send_retries 300 required_acks 0 ``` And with this I do have a lot of TIME_WAIT connections...
Hello We are having unequal events distribution of events in the partitions. our topic have 8 partitions we are noticing that one of the partition getting bulk of the events...
We are using https://github.com/takebayashi/fluent-plugin-avro to format our messages as Avro, but when sending to our kafka cluster, we need to prefix the avro data with 5 bytes to conform with...
When I restarted the kafka cluster, the following problems occurred: ``` 2020-01-17 07:29:35 +0000 [warn]: #0 failed to flush the buffer. retry_time=10 next_retry_seconds=2020-01-17 07:30:33 +0000 chunk="59c39b45987d2889fdf53f5e2d87f58c" error_class=Kafka::UnknownTopicOrPartition error="Kafka::UnknownTopicOrPartition" 2020-01-17 07:29:35...
Hi Team, I am using FluentD v0.12.42. Require help to configure the delivery semantics. Document does not explain on the exact configuration to be used. I am using cat-sweep plugin....
config: ``` @type kafka2 BROKERS username password scram_mechanism sha256 sasl_over_ssl true ssl_ca_cert [CERT] @type json @type file path /mnt/pos/buffers chunk_limit_size 64MB total_limit_size 1024MB flush_mode interval flush_interval 3s flush_thread_count 4 retry_type...
Hello, Readme https://github.com/fluent/fluent-plugin-kafka/blob/master/README.md#output-plugin says that adding `monitoring_list` would enable monitoring based on `statsd` or `datadog` for `kafka2` output plugin. I have checked the `kafka2_out.rb` and seems like it's not included,...
We use UTC timestamps when we write logs in Kafka, and date partitioning on S3. When I tried to parse a timestamp column like `{ "@t": "2020-11-11T07:58:58.7291622Z" }`, generated filename...
Good afternoon. How can I pass the password for the private key in the kafka_group input plugin. I can't find any mention of password transfer in the documentation. Example of...
I'm trying to send data from fluentd to kafka that has SASL_SSL with GSSAPI enabled. On starting the agent i see the following error with `rdkafka2` output type ``` rdkafka:...
