Admin client fails to show updated topics immediately after create_topic through list_topics

[OneRaynyDay]

Description

I'm experiencing a flaky behavior where upon AdminClient(...).create_topics([NewTopic(topic, ...)]) and I immediately AdminClient.list_topics(timeout=-1), I don't see the topic. I suspect one of the following is happening:

• create_topics not guaranteed to have actually created the topic (i.e. non-blocking)
• Topics are accurately added to the same broker but I may be calling list_topics from other broker hosts which don't have the converged, updated information
• Zookeeper has updated the information during create_topics but there's some caching layer above zookeeper which was stale for other brokers.

I might be wrong. Any info would be greatly appreciated!

How to reproduce

I am currently using Kafka 2.8.0 installed from this mirror. The client I am using is confluent-kafka==1.7.0.

>>> confluent_kafka.libversion()
('1.7.0', 17236223)
>>> confluent_kafka.version()
('1.7.0', 17235968)

Checklist

Please provide the following information:

• [x] confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()): provided above
• [x] Apache Kafka broker version: provided above
• [x] Client configuration: {"bootstrap.servers": "hosta,hostb,hostc"}. No additional settings.
• [x] Operating system: Debian linux

[edenhill]

This is (unfortunately) expected as the create_topics and list_topics requests may go to different brokers and it will take some time for a new topic to be propagated through the cluster.

[OneRaynyDay]

Thanks for explaining. I am mostly doing this in testing where I RAII my topics. Theoretically if I were to go through the same brokers for both operations would it be deterministically correct?

[pranavrth]

You can't force the requests to go to the same broker.

Closing this issue as the question has been answered. Feel free to open it if required.