ruby-kafka

ruby-kafka copied to clipboard

If this is a bug report, please fill out the following:

• Version of Ruby: 2.6
• Version of Kafka: do not know
• Version of ruby-kafka: 1.3.0

I have an issue of infiinte re-sending batch of messages if any message in batch causes MessageSizeTooLargeException.

I use async producer. In this case, the batch is re-sending as a whole , and it continues forever.

As a result, some messages are delivered more than once, thousands of times

Handling MessageSizeTooLargeException correctly is tricky, and we've yet to find a single, one-size-fits-all solution. I would advice trying to find the message that's causing the problem and skipping it. Unfortunately, the error sent back by the Kafka broker doesn't indicate which message was too large.

Issue has been marked as stale due to a lack of activity.