confluent-kafka-python icon indicating copy to clipboard operation
confluent-kafka-python copied to clipboard

Consumer. incremental_assign with different group id

Open almaz1213 opened this issue 3 years ago • 2 comments

How can I add and remove topic from consumption with different group id?

almaz1213 avatar May 21 '22 17:05 almaz1213

consumers are scoped to a particular group. you need a different consumer instance.

mhowlett avatar May 23 '22 14:05 mhowlett

@mhowlett

in addition I would recommend confluent C lib developers to:

  1. use one connection for consumer, producer and admin. why do we need establish many socket connection from only node/app/service
  2. in consumer group.id must be related on topic, not on consumer initialisation. because consumption from topic related on partitions of the topic. so, it should look smth like consumer.subscribe(topic, groupId), consumer.unsubscribe(topic). otherwise (now) we must create new consumer instance for each topic with different groups that keeps new socket connection - not good architecture

almaz1213 avatar May 23 '22 14:05 almaz1213