Delay when calling Producer.flush the first time after calling Producer.produce

[tribeiro]

Description

This issue is the same as #1581. With confluent_kafka >2 a simple script to produce kafka message experiences a 1s delay when calling Producer.flush the first time after a Producer.produce.

How to reproduce

Run the attached code , after setting KAFKA_BROKER_ADD appropriately.

minimal_write.py.tgz

You can unpack it with tar zxvf minimal_write.py.tgz.

Then run python minimal_write.py.

Checklist

Please provide the following information:

• [x] confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()):

  confluent_kafka.version(): ('2.5.0', 33882112) confluent_kafka.libversion(): ('2.5.0', 33882367)

  Installed with conda.

• [x] Apache Kafka broker version:

  Running kafka with Strimzi docker image: quay.io/strimzi/kafka:0.36.1-kafka-3.5.1

  Docker compose configuration for the kafka cluster: docker-compose.yaml.tgz

• [x] Client configuration: {...}

  There is no client but the producer configuration is

     {
         "acks": acks,
         "queue.buffering.max.ms": 0,
         "bootstrap.servers": KAFKA_BROKER_ADDR,
         "api.version.request": True,
     }
  

• [x] Operating system:

  Tested on Mac OS and Linux Alma8.

• [ ] Provide client logs (with 'debug': '..' as necessary)

• [ ] Provide broker log excerpts

• [ ] Critical issue