reactive-interaction-gateway

reactive-interaction-gateway copied to clipboard

Per @chadmott note in #165

if your schemes are designed to be fully compatible, which ours are, we have the services poll the registry every five minutes and take latest schema. This works for us, but we are small team and have tight control over the schemas and services.

That said the better way to do it would be to use a schemaAtVersion/N, and have an endpoint in each service that accepts that call and updates it’s schema reference to a version number. Update the schema and then execute the REST call to the microservice to use it.

We have to also support schema evolution for Avro in some way.

Some observations from the docs:

A schema has a unique ID and a version, but when a schema is updated, it is assigned a new unique ID (and a version)

=> For our purposes, we can ignore "version"

Relevant to the implementation:

• fetching a schema by subject: Schemex.latest/2
• fetching a schema by ID: Schemex.schema/2

Schema evolution is done according to compatibility types; they define what can change and whether consumer or producers need to be upgraded first.

By example, assuming a schema should be upgraded and there is a frontend that supports the current version:

what changed	compatibility type	what is updated first	effect	versions RIG must support
A new optional field is added.	BACKWARD	consumers	Even if the old frontend wouldn't be updated first, as a tolerant reader the frontend would simply ignore the additional field.	latest
A (non-optional) field is removed.	BACKWARD	consumers	The old frontend needs to be upgraded first, so it no longer depends on the deprecated field.	latest
A new (non-optional) field is added.	FORWARD	producers	The old frontend doesn't use the new field, so it'll simply ignore it.	latest
An optional field is removed.	FORWARD	producers	The old frontend might have relied on the optional field - it's optional when encoded in Avro, but translated to schema-less JSON any optional field becomes effectively mandatory. In other words, to a frontend this is a breaking change. Naturally, backend developers might not know that or forget about it as the Confluent docs clearly state they should upgrade producers first.	latest, if the frontend team keeps track which fields are considered optional; or any, if the frontend team doesn't know about the schema that underlies the JSON event they see.
Delete the whole thing and start over.	NONE	?	?	any

If we're gonna support the NONE type, there's not really a lot we can do other than supporting all versions there are.