aiokafka icon indicating copy to clipboard operation
aiokafka copied to clipboard

[QUESTION] What is the difference between robinhood-aiokafka and aiokafka?

Open patkivikram opened this issue 3 years ago • 0 comments

AFAIK based on this issue the main differences are:

Consumer: Request rebalance if topics were marked as missing, instead of just hanging. Kafka has a auto-create topics that kicks in after you have joined the group, but for some reason the kafka client does not request a rejoin and just hangs.

Producer: MutliTXNProducer For exactly-once we have modified the producer to support handling more than one transaction at once. If we don't have this we would need to use one producer for every topic partition.

If the worker is subscribed to 10 topics with 100 partitions each that would mean starting 1000 producer instances, each which will have as many sockets open as there are brokers...

Are these even valid now? I believe aiokafka now supports transactions after v 0.5

patkivikram avatar Oct 26 '20 16:10 patkivikram