About creating consumers

[cxin-william]

c = Consumer({ 'bootstrap.servers': servers, 'group.id': groupid, 'auto.offset.reset': offsettype, # 'auto.offset.reset': 'latest', }) c.subscribe([topic]) When I created the consumer, I thought it would be obvious if there was no data because KAFKA was not connected

Checklist

Please provide the following information:

• [ ] confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()):
• [ ] Apache Kafka broker version:
• [ ] Client configuration: {...}
• [ ] Operating system:
• [ ] Provide client logs (with 'debug': '..' as necessary)
• [ ] Provide broker log excerpts
• [ ] Critical issue

[jliunyu]

Could you please clarify your questions and fill the above checklist?

[cxin-william]

What I mean by this is that kafka should be reported when there is a problem with the connection, otherwise the data is empty and it is easy to confuse whether the data is empty or not connected

[jliunyu]

Did you meet some special case that connecting failed but without reporting any error? I just tried with your config and saw this error:

%3|1644308485.667|FAIL|rdkafka#consumer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Connect to ipv4#127.0.0.1:9092 failed: Connection refused (after 0ms in state CONNECT)
jliu@C02DM32YMD6T confluent-kafka-python % 

[cxin-william]

For example, fill in the wrong topic

[jliunyu]

I tried with the go client, found that if the topic is not exist, the error is Error: Broker: Unknown topic or partition: Subscribed topic not available: topic12: Broker: Unknown topic or partition for consumer rdkafka#consumer-1

But for python clients, if the topic the wrong, we are not able to capture it, even error_cb not works.

We are looking into it.

[edenhill]

You should see the same thing with the Python client by inspecting msg.error() of polled messages.

[edenhill]

Also see https://github.com/edenhill/librdkafka/wiki/FAQ#is-kafka-up