kafka-rest icon indicating copy to clipboard operation
kafka-rest copied to clipboard

JSON messages produced to protobuf topics fail after evolution

Open noahseger opened this issue 2 years ago • 1 comments

This is actually a problem with the schema utils not allowing or default to ignoringUnknownFields: https://github.com/confluentinc/schema-registry/blob/20a98f3c4af1ffd804d8e41a2f274d48b1418b97/protobuf-provider/src/main/java/io/confluent/kafka/schemaregistry/protobuf/ProtobufSchemaUtils.java#L103-L108

What we expect: When producing protobufs as JSON with new (compatible) fields to Kafka, those fields are ignored before the latest registered schema is evolved to include them

What happens: Bad Request: Cannot find field: XXX in message

noahseger avatar Mar 20 '24 21:03 noahseger

Exactly where this problem occurs in REST proxy: https://github.com/confluentinc/kafka-rest/blob/master/kafka-rest/src/main/java/io/confluent/kafkarest/controllers/SchemaRecordSerializerImpl.java#L123

noahseger avatar Mar 20 '24 21:03 noahseger