kafka-operator icon indicating copy to clipboard operation
kafka-operator copied to clipboard

Might schema management be in scope?

Open solsson opened this issue 6 years ago • 1 comments

This project is a very interesting approach to what I've been looking for in https://github.com/Yolean/kubernetes-kafka/issues/101. Have you also considered managing schemas?

A topic declaration could contain Avro schema(s) and the operator could create them in Schema Registry using the -key and -value convention.

Also it seems Confluent won't support JSON schema. A REST endpoint for looking up schemas, and schema format, could be useful for those writing their own Serdes.

I started a PoC operator hack a while ago, with a CRD for topic instead of a config map. How about that?

solsson avatar Dec 10 '17 18:12 solsson

We currently don't use Avro schemas as we use protobuf internally, but I've heard this request already. I think it is definitely something interesting for people using them.

Personally I think the CRDs are way forward. There's some code in operator which already works with CRDs, but it has to be hardwired manually https://github.com/nbogojevic/kafka-operator/blob/master/src/main/java/nb/kafka/operator/KafkaTopicCustomResourceManager.java

One facility with ConfigMap is that you can create them from existing properties file, not sure though that it is a standard use case.

nbogojevic avatar Dec 20 '17 11:12 nbogojevic