Support for multiple schemas in one topic

[golfpreben]

Description

Support for multiple schemas in one topic for JAVA avro serializer was added in pull request: https://github.com/confluentinc/schema-registry/pull/680 Any chance this could be supported from python too?

How to reproduce

In tried to add value.subject.name.strategy to the config of the avro producer

avroProducer = AvroProducer({
'bootstrap.servers': '127.0.0.1:9092',
'schema.registry.url': 'http://127.0.0.1:8081',
'value.subject.name.strategy': 'io.confluent.kafka.serializers.subject.TopicNameStrategy'
}, default_key_schema=key_schema, default_value_schema=value_schema)

and got the following errror:

cimpl.KafkaException: KafkaError{code=_INVALID_ARG,val=-186,str="No such configuration property: "value.subject.name.strategy""}

Checklist

Please provide the following information:

• [ ] confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()):
• [ ] Apache Kafka broker version:
• [ ] Client configuration: {...}
• [ ] Operating system:
• [ ] Provide client logs (with 'debug': '..' as necessary)
• [ ] Provide broker log excerpts
• [ ] Critical issue

[OneCricketeer]

See #442 & #401

[cschaible]

Are there any news regarding this issue? Both referenced issues (#442, #401) are closed. The issue #401 references #502, where nothing happens since months. As written in #560 the issue #502 might be included in the next release but (when) will this happen? Maybe @mhowlett can provide information, since he's reviewing the pull-request #502. Thanks in advance!

[enima2684]

Is there any workaround we can use while waiting for the new release ? Many thanks

[mhowlett]

AvroProducer is being replaced by the new avro serdes which support naming strategies: https://github.com/confluentinc/confluent-kafka-python/blob/master/confluent_kafka/schema_registry/avro.py

we don't yet support this: https://www.confluent.io/blog/multiple-event-types-in-the-same-kafka-topic/

[enima2684]

Thanks for the quick reply and good to know this functionality is already implemented.

However, it is not yet clear for me where/how to enter this configuration parameter ?

For the moment, I create a Producer object as follows :

        producer = AvroProducer(
            {
              'bootstrap.servers': "myBootstrapServers",
              'schema.registry.url': "mySchemaRegistryUrl",
              'enable.idempotence': 'true'
            },
            default_key_schema=self.get_key_schema(),
            default_value_schema=self.get_value_schema()     
 )

Is it something to add in the producer config ?

[amineouba]

did you find a solution ?