Add support for Avro - schema hosted only on Schema Registry

[alexandraprisecaru]

Hello,

I created an issue a pretty long time ago, back then I had some problems running Streamiz properly. In the meantime, I managed to get it running, something was misconfigured on my side.

As mentioned before, we're interested in decoupling the schema from the model. I created a new class, HostedSchemaAvroSerDes which is very similar to SchemaAvroSerDes, but uses Chr.Avro that enables us to fetch the schema from Confluent Schema Registry instead of keeping it in code.

I've been testing a lot lately and with the current changes I couldn't get joins to work. I debugged and noticed that in a lot of cases after joining two topics, the topic on the context was the second topic instead of the result topic.

I think that if we have the schema hard-coded locally this is not a problem, but since I want to validate with the hosted schema I need to have the correct topic, otherwise, it crashes. For this, I pass the join topic as a parameter until it reaches the ProcessorContext. I set it as optional everywhere to ensure it won't affect the current changes and I use it only in the WrappedStateStore when getting the SerializationContext.

There, if I identify a match for my value type in the Processor join topics I set it on the serialization context, otherwise it goes on the same flow it did until now.

[alexandraprisecaru]

Hi, @LGouellec! Do you think we could incorporate these changes? Please let me know if you want me to make any changes or if you have any feedback or questions.

Thanks :)

[LGouellec]

Hi @alexandraprisecaru,

I have to take time to review your PR. Sure, I will check ASAP. I will update my review status in comments.

Thanks for your contribution

[LGouellec]

I'm going to review your PR.

Just few changes :

• Move the HostedSchemaAvroSerDes class into a new package : Streamiz.Kafka.Net.SchemaRegistry.SerDes.Chr.Avro
• Can you add a sample inside samples folder with your use case, a docker-compose sample and so on

Thanks,

[alexandraprisecaru]

Hi, @LGouellec!

Sorry for the delay. I added a sample and moved the HostedSchemaAvroSerDes class to another package (Streamiz.Kafka.Net.SchemaRegistry.SerDes.Chr.Avro) :)