How to produce the undelivered messages after timeout of producer.flush(1.0)

[shuvohishab]

Description

I'm producing some messages in our Kafka server, but getting acknowledgement of message delivery(produccer.flush()) is taking a huge time. That's why I added timeout of 1 second. But I noticed that there are some messages left to deliver from our end. How to produce such undelivered messages? Any ways?

How to reproduce

producer.poll(0.0) producer.produce( topic=topic, key=str(uuid4()), value=output_msg, on_delivery=delivery_report_wrapper, headers=logger_extra, ) pending_messages = producer.flush(1.0) logger.info(f"number of messages still in queue: {pending_messages}")

Checklist

Please provide the following information:

• [ ] confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()): confluent_kafka==1.9.0
• [ ] Apache Kafka broker version:
• [ ] Client configuration: {...}
• [ ] Operating system: Linux
• [ ] Provide client logs (with 'debug': '..' as necessary): number of messages still in queue: 1
• [ ] Provide broker log excerpts
• [ ] Critical issue

[pranavrth]

I see that you are using an old version of the client. Can you use the latest one?

Can you also provide the debug logs for the same by setting debug: broker,topic,msg?

[shuvohishab]

Hi @pranavrth, thanks for your quick response. I'll come back with your answer. But can I know, do I have to update my python protobuf for this? Or, just updating confluent-kafka will work?

[pranavrth]

I think just confluent_kafka should work.

[MSeal]

Closing as no new information on trying new kafka version was provided