hermann icon indicating copy to clipboard operation
hermann copied to clipboard

Producer#push with an array of messages should call rd_kafka_produce_batch

Open rtyler opened this issue 10 years ago • 1 comments

Instead of calling rd_kafka_produce through Herman::Lib::Producer#push_single in a tight loop, we should call out to rd_kafka_produce_batch which is a better suited batch API.

rtyler avatar Aug 30 '14 22:08 rtyler

I don't know about this. My understanding of batch sends is meant only if you set the producer to be in async mode, i.e. producer.type = async (https://kafka.apache.org/documentation.html#producerconfigs).

Default producer is set to sync, if the producer is async you must also be careful of

queue.buffering.max.ms
queue.buffering.max.messages
queue.enqueue.timeout.ms
batch.num.messages

trane avatar Sep 05 '14 18:09 trane