Avro schemas are not supported in the components/schemas section

[MichaelDavisSolace]

trafficstars

Describe the bug

Asyncapi supports Avro schemas, but only if they are in the message/payload section. It would be nice if we could put Avro schemas in the components/schemas section.

To Reproduce

Steps to reproduce the behavior:

1. Create a document with an Avro schema in components/schemas. It will fail in the playground.

Expected behavior

The document should parse as it does when the schema is in the message/payload section.

Sample document

asyncapi: 2.0.0
info:
  title: Example with Avro
  version: 0.1.0
channels:
  example:
    publish:
      message:
        schemaFormat: 'application/vnd.apache.avro;version=1.9.0'
        payload:
          $ref: '#components/schemas/mySchema'
components:
  schemas:
    mySchema:
      type: record
      doc: User information
      fields:
        - name: displayName
          type: string

[github-actions[bot]]

Welcome to AsyncAPI. Thanks a lot for reporting your first issue. Please check out our contributors guide and the instructions about a basic recommended setup useful for opening a pull request.

Keep in mind there are also other channels you can use to interact with AsyncAPI community. For more details check out this issue.

[magicmatatjahu]

Thanks for the issue!

The problem as I see is that currently schemaFormat can only be used for payload. Using schemaFormat in schemas would entail creating something like this:

components:
  schemas:
    mySchema:
      schemaFormat: 'application/vnd.apache.avro;version=1.9.0'
      schema:
        type: record
        doc: User information
        fields:
          - name: displayName
            type: string

The solution for me is bad (in sense of user experience) because then user would have to write something like this if he wanted to refer to schema:

payload:
  $ref: '#components/schemas/mySchema/schema'

Another suggestion would be to use a map for custom schemaFormats, when {key} should be name of corresponding schema in components/schemas and {value} should be format.

components:

  schemaFormats:
    mySchema: 'application/vnd.apache.avro;version=1.9.0'
    
  schemas:
    mySchema:
      type: record
      doc: User information
      fields:
        - name: displayName
          type: string

TBH I don't know how it can be handled in proper way. Let's wait for other people and their insights :)

[magicmatatjahu]

I thought a little more about problem and the schemaFormat inside the schema shouldn't cause any problems:

components:
  schemas:
    mySchema:
      schemaFormat: 'application/vnd.apache.avro;version=1.9.0'
      type: record
      doc: User information
      fields:
        - name: displayName
          type: string

By default it will be JSON Schema (aka AsyncAPI Schema) and in the tooling we can parse and validate schemas using schemaFormat.

[MichaelDavisSolace]

Yes, I like that.

[magicmatatjahu]

Also we can add to the spec defaultSchemaType field in root, with functionality very similar like defaultContentType to determine the global schemaType and then the user would not have to specify the type (like avro) every time if wants to use it in all schemas.

asyncapi: 2.0.0
info: ...

defaultSchemaType: 'application/vnd.apache.avro;version=1.9.0'

components:
  schemas:
    mySchema:
      type: record
      doc: User information
      fields:
        - name: displayName
          type: string

[magicmatatjahu]

Related task: https://github.com/asyncapi/shape-up-process/issues/56

[jonaslagoni]

Sounds to me like a solid suggestion 👍 Unless we can detect these types of schemas in the parser, so you don't have to define anything 🤔 But that might be farstretched

[magicmatatjahu]

@jonaslagoni Also I thought about that, but currently supported schemaTypes (json schema, avro and raml) have different definition for type field. In Json Schema you have by default additionalProperties set to true, so you can "write" avro definition in JSON Schema if you want 😄 Also I see with automatic detection problem with custom schemaTypes - you can define and pass them in parser, so your schema can be very similar to e.g. avro but it won't be avro.

[magicmatatjahu]

Regarding to https://github.com/asyncapi/shape-up-process/issues/56 I have to conclude, a solution with schemaFormat inside a scheme will not work for all types of schema. Raml, Avro and JSON Schema can be described with JSON (including Yaml). For schema types like xsd, graphql or protobuf, i.e. for a schema that is based on its own SDL, it will have to be write in the form of multi-line raw strings like:

components:
  schemas:
    # Schema described by protobuf
    mySchema: |
      message Person {
        required string name = 1;
        required int32 id = 2;
        optional string email = 3;
      }

so we must think about another solution. Maybe this mapping with schemaFormats isn't a bad idea 🤔

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity :sleeping: It will be closed in 60 days if no further activity occurs. To unstale this issue, add a comment with detailed explanation. Thank you for your contributions :heart:

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity :sleeping: It will be closed in 60 days if no further activity occurs. To unstale this issue, add a comment with detailed explanation. Thank you for your contributions :heart:

[magicmatatjahu]

Proposal to solve the issue -> https://github.com/asyncapi/spec/issues/622