Speed of transfer data from producer to consumer by kafka broker

[Govraskirill]

Good day, dear friends. I have the next question, namely: i want to receive data from ip-camera, in kafka producer analyze and send to kafka broker topic. After this by kafka consumer open in browser. But I have problem with speed of transfer data from source to kafka broker. For example, i have delay approximately 10-15 seconds between read data and send to kafka broker and open in browser (in kafka consumer). You can find project in my repo (optimise_producer - producer, parser in new version i didin't use at the moment, consumer). How i understood i should use extra database like Confluent for save/read data so that use in producer? And second question: which tips can i use so that increase speed of send data to kafka broker and decrease delay of this action?

[OneCricketeer]

Confluent isn't a database, and wouldn't fix your problem.

Kafka isn't really best used for image/file transfer, but for a quick fix to your issue, you need to ensure you're flushing every single message to minimize producer delay, otherwise, records will accumulate at the producer before they actually reach the broker or consumer