aiokafka icon indicating copy to clipboard operation
aiokafka copied to clipboard

UnknownTopicOrPartitionError cause infinite loop inside aiokafka.consumer.group_coordinator GroupCoordinator class, commit_offsets function

Open moranmoshe opened this issue 9 months ago • 0 comments

Describe the bug I am using async consumer with subscribe by pattern. I need to support working against topics which may added and removed over time. When we remove a topic, an UnknownTopicOrPartitionError catch inside the commit_offsets function, since it is retriable, the function sleeps and then retries again (forever)

Expected behaviour Between each retry (exponential back-off) the system will force metadata update on the consumer. After few retries, the system will raise a different error (which is not retriable) to up in the chain until it is gets to the source caller.

Environment (please complete the following information):

  • aiokafka version: 0.12.0
  • Kafka Broker version: NA
  • Other information: NA

Reproducible example

# Add a short Python script or Docker configuration that can reproduce the issue.

moranmoshe avatar Feb 26 '25 15:02 moranmoshe