UnrecognizedBrokerVersion: Current client is 1.3.5, Latest client is 1.4.2

[dsoprea]

This started happening last week. The client has not changed versions. The server has not changed versions. All of the builds are occurring in identical environments, for all intents and purposes. We're running a high-volume of builds every day and it's happening once every four or five hours.

The server is on 2.12 .

Any ideas?

[jeffwidman]

Are you using KafkaClient directly? In particular, this method: http://kafka-python.readthedocs.io/en/master/apidoc/KafkaClient.html#kafka.client.KafkaClient.check_version

Do you have a copy of the stacktrace?

Also, 2.12 is the Scala version, not the broker version. Do you know what version of the broker you're on?

And what is your kafka-python config values? (feel free to obfuscate them)

[dsoprea]

Yes, to both:

  File "/var/lib/jenkins/workspace/nova_extplat_userdebug_integration/buildagent/venv/lib/python2.7/site-packages/kafka/producer/kafka.py", line 362, in __init__
    **self.config)
  File "/var/lib/jenkins/workspace/nova_extplat_userdebug_integration/buildagent/venv/lib/python2.7/site-packages/kafka/client_async.py", line 219, in __init__
    self.config['api_version'] = self.check_version(timeout=check_timeout)
  File "/var/lib/jenkins/workspace/nova_extplat_userdebug_integration/buildagent/venv/lib/python2.7/site-packages/kafka/client_async.py", line 828, in check_version
    version = conn.check_version(timeout=remaining, strict=strict)
  File "/var/lib/jenkins/workspace/nova_extplat_userdebug_integration/buildagent/venv/lib/python2.7/site-packages/kafka/conn.py", line 992, in check_version
    raise Errors.UnrecognizedBrokerVersion()
kafka.errors.UnrecognizedBrokerVersion: UnrecognizedBrokerVersion

Here's how I'm instantiating:

p = kafka.KafkaProducer(
        bootstrap_servers=hosts,
        retries=workflow.config.metric.KAFKA_RETRIES,
        value_serializer=lambda m: json.dumps(m).encode('ascii'))

The team member in charge of Kafka gave me that version number and then suddenly upgraded Kafka afterward. I'm not sure if he's going to know/recall the correct version, but I've asked.

[dsoprea]

The version having the problem is 2.12-0.11.0 .

[jeffwidman]

This doesn't quite make sense to me... you say you're using KafkaClient, but then show code using KafkaProducer...

Does your code ever directly call KafkaClient.check_version()?? Or is it just that the stacktrace contains that because KafkaProducer calls it when bootstrapping...?

[jeffwidman]

Looking at this again, now that I know more of how the code works, what's failing here is the version probing upon initial bootstrapping.

I'm very curious what the broker is returning that's causing this to fail... Should be relatively easy to check / verify this bug report using a newer cluster at some point, I just don't have time tonight.

[mvs5465]

@jeffwidman were you able to determine the cause of this?

[UmangChheda]

I am getting following traceback when trying to connect to Confluent Kafka

Traceback (most recent call last): File "kafka-consumer.py", line 7, in <module> value_deserializer=lambda m: json.loads(m.decode('utf-8'))) File "C:\Users\Umang\AppData\Local\Programs\Python\Python37-32\lib\site-packages\kafka\consumer\group.py", line 354, in __init__ self._client = KafkaClient(metrics=self._metrics, **self.config) File "C:\Users\Umang\AppData\Local\Programs\Python\Python37-32\lib\site-packages\kafka\client_async.py", line 240, in __init__ self.config['api_version'] = self.check_version(timeout=check_timeout) File "C:\Users\Umang\AppData\Local\Programs\Python\Python37-32\lib\site-packages\kafka\client_async.py", line 908, in check_version version = conn.check_version(timeout=remaining, strict=strict, topics=list(self.config['bootstrap_topics_filter'])) File "C:\Users\Umang\AppData\Local\Programs\Python\Python37-32\lib\site-packages\kafka\conn.py", line 1228, in check_version raise Errors.UnrecognizedBrokerVersion() kafka.errors.UnrecognizedBrokerVersion: UnrecognizedBrokerVersion

[jeffwidman]

This has often been caused by a SSL/SASL issue... see https://github.com/dpkp/kafka-python/issues/1796

[sandervh14]

I had this issue too, not because of SSL/SASL issues, but because my kafka-python code was running as a docker container, starting up at about the same time as the bitnami/kafka docker container.

The kafka-python code then crashed with UnrecognizedBrokerVersion because the kafka container had just not completed its startup operations.

I solved this quick and dirty for now with a simple time.sleep() before my KafkaConsumer() call. Cleaner solution is repeating a try...except until the KafkaConsumer() call isn't crashing anymore, I guess.