aiokafka
aiokafka copied to clipboard
Pass more information to the serializers/deserializers
Describe the solution you'd like
Serializers and deserializers should receive more context about the record they are working on :
- the headers
- the topic
It is especially useful when consuming from multiple topics at the same time, or topic with polymorphic messages advertised in the headers.
All information should be available (example here https://github.com/aio-libs/aiokafka/blob/master/aiokafka/consumer/fetcher.py#L286 and here https://github.com/aio-libs/aiokafka/blob/master/aiokafka/producer/producer.py#L352 ) but achieving backward compatibility could be tricky ? When they added headers to the java interface, they used a "default" method to ensure old implementations will still continue to work
Additional context
Java clients specs : https://kafka.apache.org/28/javadoc/org/apache/kafka/common/serialization/Serializer.html https://kafka.apache.org/28/javadoc/org/apache/kafka/common/serialization/Deserializer.html