confluent-kafka-python icon indicating copy to clipboard operation
confluent-kafka-python copied to clipboard

How to produce the undelivered messages after timeout of producer.flush(1.0)

Open shuvohishab opened this issue 7 months ago • 3 comments

Description

I'm producing some messages in our Kafka server, but getting acknowledgement of message delivery(produccer.flush()) is taking a huge time. That's why I added timeout of 1 second. But I noticed that there are some messages left to deliver from our end. How to produce such undelivered messages? Any ways?

How to reproduce

producer.poll(0.0) producer.produce( topic=topic, key=str(uuid4()), value=output_msg, on_delivery=delivery_report_wrapper, headers=logger_extra, ) pending_messages = producer.flush(1.0) logger.info(f"number of messages still in queue: {pending_messages}")

Checklist

Please provide the following information:

  • [ ] confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()): confluent_kafka==1.9.0
  • [ ] Apache Kafka broker version:
  • [ ] Client configuration: {...}
  • [ ] Operating system: Linux
  • [ ] Provide client logs (with 'debug': '..' as necessary): number of messages still in queue: 1
  • [ ] Provide broker log excerpts
  • [ ] Critical issue

shuvohishab avatar Jul 05 '24 04:07 shuvohishab