Resolved AsyncAPI 3.0 specification becomes invalid when channel.servers is dereferenced
According to the AsyncAPI 3.0 specification, the Channel Object includes a servers property, which must be a list of references. The documentation also states:
RECOMMENDED that parsers (or other software) dereference this property for a better development experience.
https://www.asyncapi.com/docs/reference/specification/v3.0.0#channelObject
However, in practice, when the specification is fully resolved (all $ref replaced with actual objects), the resulting document is no longer a valid AsyncAPI specification. This leads to issues such as:
The resolved spec cannot be rendered by asyncapi-react.
Tools expecting a valid AsyncAPI document fail because the servers property now contains objects instead of references.
Questions:
- why was this change introduced in AsyncAPI 3.0?
- is this intended behavior - that a fully dereferenced specification is not valid according to the AsyncAPI schema?
- if yes, what is the recommended approach for tools that need a resolved spec for rendering or processing?
Expected Behavior
- a resolved specification should remain valid and renderable by official AsyncAPI tools.
Welcome to AsyncAPI. Thanks a lot for reporting your first issue. Please check out our contributors guide and the instructions about a basic recommended setup useful for opening a pull request.
Keep in mind there are also other channels you can use to interact with AsyncAPI community. For more details check out this issue.
please share example AsyncAPI document with $ref and after dereferencing + specify what tool did you use for dereferencing.
as you can see, servers is an array of $ref, so ultimate document passed to docs generation, should not contain real server object - I'm guessing that now you are getting simply an error that document you have is not valid, right?
@derberg thanks for quick response. Tool that we are using, or rather trying implement resolve functionality is apidom.
example of unresolved file
asyncapi: 3.0.0
info:
title: Streetlights Kafka API
version: 1.0.0
description: "The Smartylighting Streetlights API allows you to remotely manage the city lights.\n\n### Check out its awesome features:\n\n* Turn a specific streetlight on/off \U0001F303\n* Dim a specific streetlight \U0001F60E\n* Receive real-time information about environmental lighting conditions \U0001F4C8\n"
license:
name: Apache 2.0
url: 'https://www.apache.org/licenses/LICENSE-2.0'
defaultContentType: application/json
servers:
scram-connections:
host: 'test.mykafkacluster.org:18092'
protocol: kafka-secure
description: Test broker secured with scramSha256
security:
- $ref: '#/components/securitySchemes/saslScram'
tags:
- name: 'env:test-scram'
description: >-
This environment is meant for running internal tests through
scramSha256
- name: 'kind:remote'
description: This server is a remote server. Not exposed by the application
- name: 'visibility:private'
description: This resource is private and only available to certain users
mtls-connections:
host: 'test.mykafkacluster.org:28092'
protocol: kafka-secure
description: Test broker secured with X509
security:
- $ref: '#/components/securitySchemes/certs'
tags:
- name: 'env:test-mtls'
description: This environment is meant for running internal tests through mtls
- name: 'kind:remote'
description: This server is a remote server. Not exposed by the application
- name: 'visibility:private'
description: This resource is private and only available to certain users
channels:
lightingMeasured:
address: 'smartylighting.streetlights.1.0.event.{streetlightId}.lighting.measured'
messages:
lightMeasured:
$ref: '#/components/messages/lightMeasured'
description: The topic on which measured values may be produced and consumed.
parameters:
streetlightId:
$ref: '#/components/parameters/streetlightId'
lightTurnOn:
address: 'smartylighting.streetlights.1.0.action.{streetlightId}.turn.on'
messages:
turnOn:
$ref: '#/components/messages/turnOnOff'
parameters:
streetlightId:
$ref: '#/components/parameters/streetlightId'
lightTurnOff:
address: 'smartylighting.streetlights.1.0.action.{streetlightId}.turn.off'
messages:
turnOff:
$ref: '#/components/messages/turnOnOff'
parameters:
streetlightId:
$ref: '#/components/parameters/streetlightId'
lightsDim:
address: 'smartylighting.streetlights.1.0.action.{streetlightId}.dim'
messages:
dimLight:
$ref: '#/components/messages/dimLight'
parameters:
streetlightId:
$ref: '#/components/parameters/streetlightId'
operations:
receiveLightMeasurement:
action: receive
channel:
$ref: '#/channels/lightingMeasured'
summary: >-
Inform about environmental lighting conditions of a particular
streetlight.
traits:
- $ref: '#/components/operationTraits/kafka'
messages:
- $ref: '#/channels/lightingMeasured/messages/lightMeasured'
turnOn:
action: send
channel:
$ref: '#/channels/lightTurnOn'
traits:
- $ref: '#/components/operationTraits/kafka'
messages:
- $ref: '#/channels/lightTurnOn/messages/turnOn'
turnOff:
action: send
channel:
$ref: '#/channels/lightTurnOff'
traits:
- $ref: '#/components/operationTraits/kafka'
messages:
- $ref: '#/channels/lightTurnOff/messages/turnOff'
dimLight:
action: send
channel:
$ref: '#/channels/lightsDim'
traits:
- $ref: '#/components/operationTraits/kafka'
messages:
- $ref: '#/channels/lightsDim/messages/dimLight'
components:
messages:
lightMeasured:
name: lightMeasured
title: Light measured
summary: >-
Inform about environmental lighting conditions of a particular
streetlight.
contentType: application/json
traits:
- $ref: '#/components/messageTraits/commonHeaders'
payload:
$ref: '#/components/schemas/lightMeasuredPayload'
turnOnOff:
name: turnOnOff
title: Turn on/off
summary: Command a particular streetlight to turn the lights on or off.
traits:
- $ref: '#/components/messageTraits/commonHeaders'
payload:
$ref: '#/components/schemas/turnOnOffPayload'
dimLight:
name: dimLight
title: Dim light
summary: Command a particular streetlight to dim the lights.
traits:
- $ref: '#/components/messageTraits/commonHeaders'
payload:
$ref: '#/components/schemas/dimLightPayload'
schemas:
lightMeasuredPayload:
type: object
properties:
lumens:
type: integer
minimum: 0
description: Light intensity measured in lumens.
sentAt:
$ref: '#/components/schemas/sentAt'
turnOnOffPayload:
type: object
properties:
command:
type: string
enum:
- 'on'
- 'off'
description: Whether to turn on or off the light.
sentAt:
$ref: '#/components/schemas/sentAt'
dimLightPayload:
type: object
properties:
percentage:
type: integer
description: Percentage to which the light should be dimmed to.
minimum: 0
maximum: 100
sentAt:
$ref: '#/components/schemas/sentAt'
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
securitySchemes:
saslScram:
type: scramSha256
description: Provide your username and password for SASL/SCRAM authentication
certs:
type: X509
description: Download the certificate files from service provider
parameters:
streetlightId:
description: The ID of the streetlight.
messageTraits:
commonHeaders:
headers:
type: object
properties:
my-app-header:
type: integer
minimum: 0
maximum: 100
operationTraits:
kafka:
bindings:
kafka:
clientId:
type: string
enum:
- my-app-id
and
resolved file
asyncapi: 3.0.0
info:
title: Streetlights Kafka API
version: 1.0.0
description: |
The Smartylighting Streetlights API allows you to remotely manage the city lights.
### Check out its awesome features:
* Turn a specific streetlight on/off U0001F303
* Dim a specific streetlight U0001F60E
* Receive real-time information about environmental lighting conditions U0001F4C8
license:
name: Apache 2.0
url: https://www.apache.org/licenses/LICENSE-2.0
defaultContentType: application/json
servers:
scram-connections:
host: test.mykafkacluster.org:18092
protocol: kafka-secure
description: Test broker secured with scramSha256
security:
- type: scramSha256
description: Provide your username and password for SASL/SCRAM authentication
tags:
- name: env:test-scram
description: This environment is meant for running internal tests through scramSha256
- name: kind:remote
description: This server is a remote server. Not exposed by the application
- name: visibility:private
description: This resource is private and only available to certain users
mtls-connections:
host: test.mykafkacluster.org:28092
protocol: kafka-secure
description: Test broker secured with X509
security:
- type: X509
description: Download the certificate files from service provider
tags:
- name: env:test-mtls
description: This environment is meant for running internal tests through mtls
- name: kind:remote
description: This server is a remote server. Not exposed by the application
- name: visibility:private
description: This resource is private and only available to certain users
channels:
lightingMeasured:
address: "smartylighting.streetlights.1.0.event.{streetlightId}.lighting.measured"
messages:
lightMeasured:
name: lightMeasured
title: Light measured
summary: Inform about environmental lighting conditions of a particular streetlight.
contentType: application/json
traits:
- headers:
type: object
properties:
my-app-header:
type: integer
minimum: 0
maximum: 100
payload:
type: object
properties:
lumens:
type: integer
minimum: 0
description: Light intensity measured in lumens.
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
description: The topic on which measured values may be produced and consumed.
parameters:
streetlightId:
description: The ID of the streetlight.
lightTurnOn:
address: "smartylighting.streetlights.1.0.action.{streetlightId}.turn.on"
messages:
turnOn:
name: turnOnOff
title: Turn on/off
summary: Command a particular streetlight to turn the lights on or off.
traits:
- headers:
type: object
properties:
my-app-header:
type: integer
minimum: 0
maximum: 100
payload:
type: object
properties:
command:
type: string
enum:
- "on"
- "off"
description: Whether to turn on or off the light.
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
parameters:
streetlightId:
description: The ID of the streetlight.
lightTurnOff:
address: "smartylighting.streetlights.1.0.action.{streetlightId}.turn.off"
messages:
turnOff:
name: turnOnOff
title: Turn on/off
summary: Command a particular streetlight to turn the lights on or off.
traits:
- headers:
type: object
properties:
my-app-header:
type: integer
minimum: 0
maximum: 100
payload:
type: object
properties:
command:
type: string
enum:
- "on"
- "off"
description: Whether to turn on or off the light.
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
parameters:
streetlightId:
description: The ID of the streetlight.
lightsDim:
address: "smartylighting.streetlights.1.0.action.{streetlightId}.dim"
messages:
dimLight:
name: dimLight
title: Dim light
summary: Command a particular streetlight to dim the lights.
traits:
- headers:
type: object
properties:
my-app-header:
type: integer
minimum: 0
maximum: 100
payload:
type: object
properties:
percentage:
type: integer
description: Percentage to which the light should be dimmed to.
minimum: 0
maximum: 100
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
parameters:
streetlightId:
description: The ID of the streetlight.
operations:
receiveLightMeasurement:
action: receive
channel:
address: "smartylighting.streetlights.1.0.event.{streetlightId}.lighting.measured"
messages:
lightMeasured:
name: lightMeasured
title: Light measured
summary: Inform about environmental lighting conditions of a particular streetlight.
contentType: application/json
traits:
- headers:
type: object
properties:
my-app-header:
type: integer
minimum: 0
maximum: 100
payload:
type: object
properties:
lumens:
type: integer
minimum: 0
description: Light intensity measured in lumens.
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
description: The topic on which measured values may be produced and consumed.
parameters:
streetlightId:
description: The ID of the streetlight.
summary: Inform about environmental lighting conditions of a particular streetlight.
traits:
- bindings:
kafka:
clientId:
type: string
enum:
- my-app-id
messages:
- name: lightMeasured
title: Light measured
summary: Inform about environmental lighting conditions of a particular streetlight.
contentType: application/json
traits:
- headers:
type: object
properties:
my-app-header:
type: integer
minimum: 0
maximum: 100
payload:
type: object
properties:
lumens:
type: integer
minimum: 0
description: Light intensity measured in lumens.
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
turnOn:
action: send
channel:
address: "smartylighting.streetlights.1.0.action.{streetlightId}.turn.on"
messages:
turnOn:
name: turnOnOff
title: Turn on/off
summary: Command a particular streetlight to turn the lights on or off.
traits:
- headers:
type: object
properties:
my-app-header:
type: integer
minimum: 0
maximum: 100
payload:
type: object
properties:
command:
type: string
enum:
- "on"
- "off"
description: Whether to turn on or off the light.
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
parameters:
streetlightId:
description: The ID of the streetlight.
traits:
- bindings:
kafka:
clientId:
type: string
enum:
- my-app-id
messages:
- name: turnOnOff
title: Turn on/off
summary: Command a particular streetlight to turn the lights on or off.
traits:
- headers:
type: object
properties:
my-app-header:
type: integer
minimum: 0
maximum: 100
payload:
type: object
properties:
command:
type: string
enum:
- "on"
- "off"
description: Whether to turn on or off the light.
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
turnOff:
action: send
channel:
address: "smartylighting.streetlights.1.0.action.{streetlightId}.turn.off"
messages:
turnOff:
name: turnOnOff
title: Turn on/off
summary: Command a particular streetlight to turn the lights on or off.
traits:
- headers:
type: object
properties:
my-app-header:
type: integer
minimum: 0
maximum: 100
payload:
type: object
properties:
command:
type: string
enum:
- "on"
- "off"
description: Whether to turn on or off the light.
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
parameters:
streetlightId:
description: The ID of the streetlight.
traits:
- bindings:
kafka:
clientId:
type: string
enum:
- my-app-id
messages:
- name: turnOnOff
title: Turn on/off
summary: Command a particular streetlight to turn the lights on or off.
traits:
- headers:
type: object
properties:
my-app-header:
type: integer
minimum: 0
maximum: 100
payload:
type: object
properties:
command:
type: string
enum:
- "on"
- "off"
description: Whether to turn on or off the light.
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
dimLight:
action: send
channel:
address: "smartylighting.streetlights.1.0.action.{streetlightId}.dim"
messages:
dimLight:
name: dimLight
title: Dim light
summary: Command a particular streetlight to dim the lights.
traits:
- headers:
type: object
properties:
my-app-header:
type: integer
minimum: 0
maximum: 100
payload:
type: object
properties:
percentage:
type: integer
description: Percentage to which the light should be dimmed to.
minimum: 0
maximum: 100
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
parameters:
streetlightId:
description: The ID of the streetlight.
traits:
- bindings:
kafka:
clientId:
type: string
enum:
- my-app-id
messages:
- name: dimLight
title: Dim light
summary: Command a particular streetlight to dim the lights.
traits:
- headers:
type: object
properties:
my-app-header:
type: integer
minimum: 0
maximum: 100
payload:
type: object
properties:
percentage:
type: integer
description: Percentage to which the light should be dimmed to.
minimum: 0
maximum: 100
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
components:
messages:
lightMeasured:
name: lightMeasured
title: Light measured
summary: Inform about environmental lighting conditions of a particular streetlight.
contentType: application/json
traits:
- headers:
type: object
properties:
my-app-header:
type: integer
minimum: 0
maximum: 100
payload:
type: object
properties:
lumens:
type: integer
minimum: 0
description: Light intensity measured in lumens.
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
turnOnOff:
name: turnOnOff
title: Turn on/off
summary: Command a particular streetlight to turn the lights on or off.
traits:
- headers:
type: object
properties:
my-app-header:
type: integer
minimum: 0
maximum: 100
payload:
type: object
properties:
command:
type: string
enum:
- "on"
- "off"
description: Whether to turn on or off the light.
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
dimLight:
name: dimLight
title: Dim light
summary: Command a particular streetlight to dim the lights.
traits:
- headers:
type: object
properties:
my-app-header:
type: integer
minimum: 0
maximum: 100
payload:
type: object
properties:
percentage:
type: integer
description: Percentage to which the light should be dimmed to.
minimum: 0
maximum: 100
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
schemas:
lightMeasuredPayload:
type: object
properties:
lumens:
type: integer
minimum: 0
description: Light intensity measured in lumens.
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
turnOnOffPayload:
type: object
properties:
command:
type: string
enum:
- "on"
- "off"
description: Whether to turn on or off the light.
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
dimLightPayload:
type: object
properties:
percentage:
type: integer
description: Percentage to which the light should be dimmed to.
minimum: 0
maximum: 100
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
sentAt:
type: string
format: date-time
description: Date and time when the message was sent.
securitySchemes:
saslScram:
type: scramSha256
description: Provide your username and password for SASL/SCRAM authentication
certs:
type: X509
description: Download the certificate files from service provider
parameters:
streetlightId:
description: The ID of the streetlight.
messageTraits:
commonHeaders:
headers:
type: object
properties:
my-app-header:
type: integer
minimum: 0
maximum: 100
operationTraits:
kafka:
bindings:
kafka:
clientId:
type: string
enum:
- my-app-id
Below you can see how it look in AsyncApi Studio.
Hey @derberg 👋
The same issue can be reproduced using @asyncapi/parser library
Steps to reproduce:
- Create a valid AsyncAPI 3 specs which has defined messages for channel. For example https://gist.github.com/robert-hebel-sb/5432fbc43624c20f54210261d2bdf969
- Use @asyncapi/parser to parse definition
import { Parser } from '@asyncapi/parser';
const parser = new Parser();
.
.
.
const res = await parser.parse(content);
return res.document.json()
- Paste parsed result in https://studio.asyncapi.com/ https://gist.github.com/robert-hebel-sb/3f8a7817421f6e9a1bf542059a0dbc63
Expected behavior
- parsed spec is validated and preview is available
Actual behavior
- Error occurs in validation, no preview is displayed
I see for resolved specs altered async3 schema should be used. I'm not sure against which schema validation happens for the case described here https://github.com/asyncapi/parser-js/blob/master/packages/parser/src/ruleset/functions/documentStructure.ts#L55
@team i want to work on this isuue could you please assign this to me ?
@maishivamhoo123 what will be the solution? Does it mean that this is bug in @asyncapi/parser? After fix fully resolved specification should be validated and preview should be available?
The problem is that the parser turns $refs into full objects, but the validator expects them to be references (according to the spec). This isn't an issue when validating for yourself, you should just validated unresolved schemas instead.
Now for use by other tools like asyncapi-react, I think we need to think of a long term idea for this.
My first instincts say we should let these tools accept fully resolved schemas instead of unresolved ones. Probably we need to have two different specs for 1. unresolved schemas, and 2. resolved ones. And let the tools accept resolved schemas instead.
Or if my understanding is right, even better if we could embed parsers in these tools and have a pre-validation step of parsing? And then we don't need to have two specs.
@MichakrawSB why was the issue closed?
Sorry, my mistake
hello @MichakrawSB i am new to these organization and sir can i work on these issue and can you help me how i will contribute in these organization. Thanks
You can find better places to ask such questions. Please don't add unnecessary distractions to ongoing issues.