kafka-python icon indicating copy to clipboard operation
kafka-python copied to clipboard

KafkaProducer cannot send data standalone

Open jason19970210 opened this issue 3 years ago • 1 comments

Try to compare these 2 sample below

if __name__ == '__main__':
    producer.send(topic, value=b'message')
if __name__ == '__main__':
    while 1:
        producer.send(topic, value=b'message')
        time.sleep(2)

So once standalone the producer.send, the consumer cant get the new data, but if we add the while 1 statement, then messages are run regularly.

Please have a look on the way to explain or fix !!! Thanks a lot !!!

jason19970210 avatar Dec 16 '21 06:12 jason19970210

@jason19970210 use .flush() on producer at the end.

if __name__ == '__main__':
    producer.send(topic, value=b'message')
    producer.flush()

for content KafkaProducer send message in async call, first add message in local buffer and than send it to topic. in first case your script will exit with KafkaProducer instance immediately after adding it to local buffer even before producer get change to send it to topic.

vish0l avatar Dec 17 '21 15:12 vish0l