fluent-plugin-kafka icon indicating copy to clipboard operation
fluent-plugin-kafka copied to clipboard

Published 20 hours ago • verified publisher icon fluent

out_kafka2: Broker pool to take care of fetching metadata

Open dipendra-singh opened this issue 7 months ago • 1 comments

Which issue(s) this PR fixes: https://github.com/fluent/fluent-plugin-kafka/issues/502

What this PR does / why we need it: On enabling the get_kafka_client_log true, this can be figured out the leader of partitions is being queried multiple times: here this metadata if refreshed by broker pool (here). Adding this in producer would resolve the issue. additionally, this is a part of an optimisation done on ruby-kafka.

Test cases passing post the changes.

Docs Changes: NA

dipendra-singh avatar Dec 01 '23 23:12 dipendra-singh

@ashie Requesting review here

dipendra-singh avatar Dec 16 '23 11:12 dipendra-singh