confluent-kafka-python icon indicating copy to clipboard operation
confluent-kafka-python copied to clipboard

Message.__len__ fails if the value has been deserialized to a type that does not support __len__

Open choogeboom opened this issue 4 years ago • 2 comments

Description

In the Message.__len__ method it gets the length of the value and returns that. Unfortunately, this means that if a user is using a DeserializingConsumer that deserializes the value to a type that does not implement __len__, then calling len(message) fails with the following error:

TypeError: object of type 'SomeTypeWithoutLen' has no len()

This causes errors if you try to use a message in a truthy manner, such as

if message:
    do_something()

or

actual_message = message or some_other_value

How to reproduce

  1. Create a topic
  2. Produce a message on the topic where the value is serialized using some avro schema
  3. Create a DeserializingConsumer with a value.deserializer that deserializes the message to a type that doesn't implement __len__, such as a dataclass representing the avro record. This can be accomplished with the from_dict attribute of the AvroDeserializer.
  4. Consume the message message = consumer.poll()
  5. Call len(message)

Checklist

Please provide the following information:

confluent-kafka-python: ('1.7.0', 17235968) librdkafka: ('1.7.0', 17235968)

choogeboom avatar Sep 01 '21 20:09 choogeboom

Hi @choogeboom, thanks for reporting this. I will add this to our backlog to plan to support this.

jliunyu avatar Sep 07 '21 22:09 jliunyu

Hey, apart from the mentioned issue, could be the case that the first check should be done like this? self->value != Py_None ?

josumoreno-BP avatar Feb 22 '24 15:02 josumoreno-BP