Introduction of registries at the top level

[thompson-tomo]

Message registries are becoming more and more appealing for teams to leverage as the focus on interoperability and standardised messaging grows. Based on that would I would like to see is the introduction of registries as a top level object so that they can be defined in the 1 location. The difference between a schema path & a registry is that the registry points to the file which contains 1 or more schema definitions/groups whereas schema path refers to a position within the file

The definition of a registry would need to include:

• Url
• Registry Type: enum
• Merge content: enum (none, required, all)

Then the items from the registry can be used via:

$ref: '#/registry/{{registryName}}/item/{{path}}

If the developer want to switch to a newer version, they just update the registry.

At the same the embed items flag would be useful for importing definitions of objects which are being transported via strings in other objects ie cloud events

[github-actions[bot]]

Welcome to AsyncAPI. Thanks a lot for reporting your first issue. Please check out our contributors guide and the instructions about a basic recommended setup useful for opening a pull request.
Keep in mind there are also other channels you can use to interact with AsyncAPI community. For more details check out this issue.

[dalelane]

I'd appreciate a concrete example, if you have one please? Perhaps this is obvious to others, but I'm not familiar with using schema registries in a more protocol-agnostic way, as my experience with them is quite Kafka-specific.

[thompson-tomo]

An example of what I would like to do is:

1. Add a registry entry like below

Registries:
  CloudEvent:
    Url: "https://cloudevents.io/schemas/1.0.3/JsonSchema.json"
    RegistryType: JsonSchema
    MergeContent: Required

2. When defining my message, I can simply consume the definition from the registry like so

$ref: '#/registry/CloudEvent/item/Events/CloudEventBinary'

The merge content all would be used for example in the cloud event scenario where within the payload there is data needing to be deserialised to an object.

[dalelane]

If I'm the only person who doesn't understand this, I'm happy to step back and be quiet - but I don't understand what I would do with the above.

What is the registry in your above example? Is this a cloudevent-specific thing?

What does a registry type involve? Are you assuming that registries would only store a single type of schema? Perhaps my confusion is just because the registry I work with most frequently (Apicurio - https://www.apicur.io/registry/) stores many different types of schemas. For one topic, I might be retrieving a Protobuf schema, for another topic, I'll use an Avro schema, for a third topic, I'll use a JSON schema - but they all come from the same schema registry.

As such, does it make sense to do this as a top-level definition? If you're looking to provide an external reference to a schema, would it perhaps be better to do this at a message level?

[dalelane]

I just noticed your most recent edit to the issue:

the registry points to the file which contains 1 or more schema definitions/groups

That helps clear up my misunderstanding a bit - I was assuming that "registry" was referring to a runtime component. I hadn't realised you were thinking of it as a file.

[thompson-tomo]

What is the registry in your above example? Is this a cloudevent-specific thing?

What does a registry type involve? Are you assuming that registries would only store a single type of schema?

Perhaps the name registries might not be the best choice but can't think of a better one. But yes, it would point to a single file which would contain multiple objects which could be used.

As such, does it make sense to do this as a top-level definition? If you're looking to provide an external reference to a schema, would it perhaps be better to do this at a message level?

As per my previous example you would still retain the message level reference but the path is parametised as now it leverages that common file path definition. At the same time it also needs to be global to cater for providing the schema of objects which have been serialised into a field which is transported via a field in the message as it's the case with cloud events.

Perhaps following spec would help the situation

Registries:
  CloudEvent: -> id of the registry
    Url: "https://cloudevents.io/"
    Files:
      CE: -> ID of the file
        Path: "schemas/1.0.3/JsonSchema.json"
        RegistryType: JsonSchema
        MergeContent: Required

And the message level reference would be

$ref: 'registry/CloudEvent/CE#/Events/CloudEventBinary'

[derberg]

is this basically a proposal to what is workaround with https://github.com/asyncapi/spec/issues/1102?

[thompson-tomo]

This works hand in hand with #1102 by enabling moving of the remote url out of the ref into a reusable variable and the ref contains a pointer to the variable + the path.

[derberg]

are you basically suggesting something that in JSON Schema is known as $id ?

[thompson-tomo]

Not quite as you could source items from multiple registries hence defining it as an array and referencing the registry by id aka name.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity :sleeping:

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience :heart: