kafka-python icon indicating copy to clipboard operation
kafka-python copied to clipboard

Messages exceeding the size are silently dropped

Open jacopofar opened this issue 1 year ago • 1 comments

Hello, I noticed some messages not being published to Kafka although the logs report no error, and it seems that if a message exceeds the maximum size (max_request_size in the Producer, in my case corresponding to the value in the topic settings) it is discarded but no exception is raised.

Is this a known behavior?

jacopofar avatar Feb 09 '24 11:02 jacopofar

A small update: calling flush() after send() I do get a warning:

WARNING kafka.producer.record_accumulator [Error 10] MessageSizeTooLargeError

it is a warning rather than an exception, but it's an improvement

jacopofar avatar Feb 09 '24 14:02 jacopofar