kafka-operator
kafka-operator copied to clipboard
Might schema management be in scope?
This project is a very interesting approach to what I've been looking for in https://github.com/Yolean/kubernetes-kafka/issues/101. Have you also considered managing schemas?
A topic declaration could contain Avro schema(s) and the operator could create them in Schema Registry using the -key
and -value
convention.
Also it seems Confluent won't support JSON schema. A REST endpoint for looking up schemas, and schema format, could be useful for those writing their own Serdes.
I started a PoC operator hack a while ago, with a CRD for topic instead of a config map. How about that?
We currently don't use Avro schemas as we use protobuf internally, but I've heard this request already. I think it is definitely something interesting for people using them.
Personally I think the CRDs are way forward. There's some code in operator which already works with CRDs, but it has to be hardwired manually https://github.com/nbogojevic/kafka-operator/blob/master/src/main/java/nb/kafka/operator/KafkaTopicCustomResourceManager.java
One facility with ConfigMap is that you can create them from existing properties file, not sure though that it is a standard use case.