Refactor spring-cloud-stream-schema into three artifacts, core, client, and server

[sabbyanandan]

@jmax01 commented on Tue Jul 16 2019

After working extensively with the spring-cloud-stream-schema projects I think the introduction of a spring-cloud-stream-schema-core project would be helpful.

Our team has enhanced our schema-registry instance with the following endpoints:

    public ResponseEntity<List<MimeType>> findAllMimeTypes(Optional<String> format);
    public ResponseEntity<List<MimeType>> findAllMimeTypesBySubject(final String subject);
    public ResponseEntity<List<Schema>> findAll();
    public ResponseEntity<List<Schema>> findAllByMimeType(final MimeType mimeType);
    public ResponseEntity<MimeType> findLatestMimeTypeBySubject(String subject);
    public ResponseEntity<MimeType> findMimeTypeById(final int id);
    public ResponseEntity<MimeType> findMimeTypeBySchemaDefinition(String schemaDefinition, final String format);
    public ResponseEntity<Schema> findByMimeType(final MimeType mimeType);
    public ResponseEntity<Schema> findBySchemaDefinition(final String schemaDefinition, String format);
    public ResponseEntity<Schema> findLatestByMimeType(final MimeType mimeType);

During that enhancement there was many times where I wished that common items were shared across the client and server modules, especially during testing.

---

@sabbyanandan commented on Tue Jul 16 2019

Hi, @jmax01. Thank you for bringing this up. That's a nice set of improvements to the existing set of operations. As a heads-up, starting from v3.0, we are going to take the Schema Registry component from the core to its own repo, as a standalone project. Hopefully, then, it'd be easy to add more feature value-adds and evolve it in isolation rapidly. The goal is to branch this out by the next 3.0 milestone (maybe: M3) - stay tuned.

---

@jmax01 commented on Tue Jul 16 2019

Let me just brain dump a couple things:

spring-cloud-stream-schema-server

• We had the advantage of standardizing one MimeType format where that is not the case anything that takes or returns a MimeType would have to have formatting rules (prefix+subject naming strategy) passed in the request.
• Schema.description should be lazily loaded, while not large (8k max currently) it is wasteful to pull it for actions that don't require it (delete, MimeType generation, etc). We created an entity just of the surrogate and business keys to avoid this.
• Caching. At least responses to requests that supply the version or the id should be cached.

spring-cloud-stream-schema (AKA spring-cloud-stream-schema-client) Note that we reimplemented much of the AbstractAvroMessageConverter and AvroSchemaRegistryClientMessageConverter to provide better performance, but more importantly to allow conversions without the requirement of the MimeType being supplied via the header or conversionHint. Here is the list of issues and PRs I opened during that process.

• Caching: We aggressively cache Avro Schemas, SchemaReferences, MimeTypes, subjects and GenericContainer implementing classes. Our caching begins at the canConvert* level to minimize redundant processing.
• Add the converter mentioned here: https://github.com/spring-cloud/spring-cloud-stream/issues/1296