consumer not poll message and GroupCoordinator disconnect

[xianzhengguang]

Description

run for a while，consumer not pulll message ， consumer idling process exist error:

%6|1642580757.330|FAIL|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator:: Disconnected (after 1137914ms in state UP, 1 identical error(s) suppressed) %6|1642580757.348|FAIL|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator: Disconnected (after 1137369ms in state UP, 1 identical error(s) suppressed)

How to reproduce

consumer config: self.configs = { "bootstrap.servers": bootstrap_servers, "group.id": group_id, "enable.auto.commit": False, "auto.offset.reset": "latest", 'max.poll.interval.ms': 1800000, 'session.timeout.ms': 120000 } poll:msg = self.consumer.poll(1.0)

Checklist

Please provide the following information:

• [ ] confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()):confluent-kafka python 1.7.2
• [ ] Apache Kafka broker version: kafka-1.0.1-kafka3.1.1
• [ ] Client configuration: {...}
• [ ] Operating system: CentOS Linux release 7.6.1810 (Core)
• [ ] Provide client logs (with 'debug': '..' as necessary)
• [ ] Provide broker log excerpts
• [ ] Critical issue %6|1642580757.330|FAIL|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator:: Disconnected (after 1137914ms in state UP, 1 identical error(s) suppressed) %6|1642580757.348|FAIL|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator: Disconnected (after 1137369ms in state UP, 1 identical error(s) suppressed)

[xianzhengguang]

Hope to help you soon thx

[jliunyu]

Can you please provide more debug logs so we can take a look?

[xianzhengguang]

Can you please provide more debug logs so we can take a look

[ashish-gupta1-by]

I am also facing this issue Logs

%5|1653298173.787|REQTMOUT|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/0: Timed out LeaveGroupRequest in flight (after 5004ms, timeout #0): possibly held back by preceeding blocking JoinGroupRequest with timeout in 96883ms
14:59:33.787

%3|1653298173.787|FAIL|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator: 1 request(s) timed out: disconnect (after 3696484ms in state UP)
14:59:33.787

%4|1653298173.787|REQTMOUT|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/0: Timed out 1 in-flight, 0 retry-queued, 0 out-queue, 0 partially-sent requests

conf

consumer_conf = { 'bootstrap.servers': conf.get('bootstrap_servers'), 'group.id': conf.get('consumer_group_id'), 'auto.offset.reset': 'latest', 'enable.auto.commit': False, 'sasl.mechanism': 'PLAIN', 'security.protocol': 'SASL_SSL', 'sasl.username': '$ConnectionString', 'sasl.password': conf.get('sasl_plain_password'), 'on_commit': self.commit_completed, 'max.poll.interval.ms': 3600000 }

[mhowlett]

LeaveGroupRequest - this implies you've called close on the consumer.

this is a bit difficult to investigate without more information. we'll keep this report in mind if / when we see other like issues.