kafka-go

kafka-go copied to clipboard

Greetings! I want to implement sticky partitioner for my application by implementing Balancer and almost everything seems to be just fine except one small thingy - it is impossible to figure out is batch ready or not. I think we can expose OnTriggerFunc to be called in writerBatch.trigger (if not nil), just like Completion func

If you go here and look https://www.confluent.io/blog/apache-kafka-producer-improvements-sticky-partitioner/

You'll find out that Java guys are actually exposes onNewBatch which is..kinda similar to what i'd love to see here

Hello @nmerkulov, thanks for opening the issue!

Would you be able to provide an example of how you would use the trigger?

Sure

Initially i was thinking about implementing sticky balancer Here how it should work: Stick to single partition until batch is ready (by reaching out wait time or size) Then stick to another one

And for now there is no clean way to figure - is current partition ready or not? So by exposing "call this function on trigger" i can catch readyness of batch and stick to another partition