Let channels be identified by an ID rather than their address.

[smoya]

Context

This is part of the RFC https://github.com/asyncapi/spec/issues/618#issuecomment-980093487. In an effort around reducing the scope of such RFC, it has been split into several proposals so they can move forward independently.

The problem

Channels are currently identified by their address. E.g.:

channels:
  user/signedup: # this is both the address and the identifier of the channel
    description: ...

With the current approach, and once https://github.com/asyncapi/spec/issues/660 happens, we will have few issues:

• When referencing a channel, the channel identifier should match with the refereced one, otherwise the address will be different than in the referenced channel. E.g.:

  # common.yaml
  channels:
    user/signedup:
        description: ...
  
  

  # asyncapi.yaml
  channels:
    user/signedup: # here we have to use the same identifier as in common.yaml, otherwise the address will be different
      $ref: 'common.yaml#/components/channels/user~signedup'
  

• That introduces a problem when we need to change the address of a shared channel. Let's say Kafka topic name changes. With the current approach, we need to change the address on all files that reference the user/signedup channel from common.yaml (as JSON Reference will fail as well).

• As you may noticed, the JSON Pointer for user/signedup channel is user~1signedup. It is unexpected and you have to have some JSON Pointer knowledge or find examples to find out such format. It is very error-prone.

Suggestion

To add a new address field to the channel definition that represents the real address and not the channel identifier. With that change, the previous example will look like:

# common.yaml
channels:
  userSignedUp:
      address: user/signedup
      description: ...
      

# asyncapi.yaml
channels:
  myUserSignedUp: # here we can add whatever ID we want. If address changes in `userSignedUp` from common.yaml, nothing will need to change in this file
    $ref: 'common.yaml#/components/channels/userSignedUp' # no slash transformation is needed in the JSON Pointer :)

Fixes https://github.com/asyncapi/spec/issues/739 as well

[jonaslagoni]

I love the move to a property, whether address is the right keyword, not so sure, but I don't have other alternatives at the moment.

What exactly is the reason we need an ID rather than making channels an array?

Especially now that https://github.com/asyncapi/spec/pull/665 is added.

[smoya]

I love the move to a property, whether address is the right keyword, not so sure, but I don't have other alternatives at the moment.

What exactly is the reason we need an ID rather than making channels an array?

Especially now that #665 is added.

I guess the reason is that we might also allow people to reference channels not only from components but from channels at root level. Another thing is that you need to differentiate between servers anyway, for example, when generating code or documentation. Let's say I want to generate a websocket client but in the document, two servers are declared. How can I pick up one or another? By it's ID.

[jonaslagoni]

Okay I retract my suggestion, you are right.

Especially if you want to define two channels operating over the same address, with slightly different behavior, you would need to add a channelId instead to differentiate them, if we had arrays.

[smoya]

The following PR's have been created:

• https://github.com/asyncapi/spec/pull/719
• https://github.com/asyncapi/spec-json-schemas/pull/171
• Pending Parser-JS work.

[char0n]

Hi everybody,

I was CCed in https://github.com/asyncapi/spec-json-schemas/pull/171, so let me share my opinion as well here. IMHO I can see a couple of problems here:

Components.channels field in defined in following way:

This means that string that forms the key can be arbitrary and doesn't have any semantics. string !== {channel}. The proof lies in following AsyncAPI definition fragment:

components:
  channels:
    channel1:
      description: channel description

channels:
  user/signup:
    $ref: "#/components/channels/channel1"

As you can clearly see, the channel1 is just an arbitrary identifier of the channel and has nothing to do with {channel} (path).

The only place where this is problematic is in following fragment:

channels:
 user/logout:
   $ref: "#/channels/user~1signedup"
 user/signedup:
   description: channel description

But as we now have reusable Components.channels field, this usecase should be discouraged and anything that is intended to be reusable should be in Components.channels. This encourages using best practices and putting stuff where it belongs. IMHO I don't think this proposal is needed (don't hate me:).

[magicmatatjahu]

@char0n

But as we now have reusable Components.channels field, this usecase should be discouraged and anything that is intended to be reusable should be in Components.channels. This encourages using best practices and putting stuff where it belongs. IMHO I don't think this proposal is needed (don't hate me:).

Please look at this proposal from point of view person who need to write several AsyncAPI documents and use that same channel in few documents. Of course can make reference by:

$ref: "#/components/channel1"

but then needs to remember that (e.g.) "user/signup" channel address, yes? :) So that PR adds possibility to make more easier the "channels registry" and treats address as channelID :)

Please read https://github.com/asyncapi/spec/issues/618 proposal to understand why we wanna make it :) Of course "channel registry" is not the only reason, but it is one of many, perhaps the best why it makes sense.

[char0n]

Please look at this proposal from point of view person who need to write several AsyncAPI documents and use that same channel in few documents. Of course can make reference by:

@magicmatatjahu I really did that. Where I want to get is to have AsyncAPI spec version where the only required fields are asyncapi and info. That way I can create reusable definitions that only contain stuff that is used in multiple other definition. It would look like this:

reusable.yaml

asyncapi: 2.3.0
info:
  title: reusable document fragment
  version: 1.0.0
components:
  channels:
    channel1:
      description: channel description

Then it will be consumed by other definitions:

definition1.yaml

asyncapi: 2.3.0
info:
  title: definition 1
  version: 1.0.0
channels:
  path/to/channel:
    $ref: "./reusable.yaml/#components/channels/channel1"

definition2.yaml

asyncapi: 2.3.0
info:
  title: definition 2
  version: 1.0.0
channels:
  path/to/channel:
    $ref: "./reusable.yaml/#components/channels/channel1"

This is currently not possible as AsyncAPI.channels field is currently required (this requires small backward compatible change to support) . My opinion on this is influenced by heavy usage of OpenAPI and doing exactly the stuff that I describe here using OpenAPI.

Please read https://github.com/asyncapi/spec/issues/618 proposal to understand why we wanna make it :) Of course "channel registry" is not the only reason, but it is one of many, perhaps the best why it makes sense.

#618 is super long ;] I'm trying to look at this change introduced in this issue in isolated way. People have full ability to create keys as they seem fit and can use a lot of imagination to come up with the key names that are sometimes exotic. This issue IMHO tries to resolve one particular issue which is resolvable by other means (small change in spec + recommended practices).

[magicmatatjahu]

This is currently not possible as AsyncAPI.channels field is currently required (this requires small backward compatible change to support) . My opinion on this is influenced by heavy usage of OpenAPI and doing exactly the stuff that I describe here using OpenAPI.

Please read https://github.com/asyncapi/spec/issues/618 proposal to understand why we wanna make it :) Of course "channel registry" is not the only reason, but it is one of many, perhaps the best why it makes sense.

https://github.com/asyncapi/spec/issues/618 is super long ;] I'm trying to look at this change introduced in this issue in isolated way. People have full ability to create keys as they seem fit and can use a lot of imagination to come up with the key names that are sometimes exotic. This issue IMHO tries to resolve one particular issue which is resolvable by other means (small change in spec + recommended practices).

Yeah, exactly, currently it's not possible. Changing it is easy (to make channels options) but we need for that 3.0.0.

Maybe I don't understand the second paragraph (sorry!), but a channel key and an address are two completely different things in my opinion. I understand that the keys may be different, but this should be just "information" for the developer, not for a user reading the documentation. It's a bit like a server url - it may have a different key but url identifies its "uniqueness" and so it is with the channel address.

recommended practices

And this is probably the problem, because it is better to solve this at the level of specification and not through recommended practices. There are things that should be recommended, but if there is an option to fix it in the specification itself, then let's go the second way. As I wrote, currently the problem are references and people then have to remember that a given channel should have a given key in the .channels field (exactly address).

Of course, lets wait for others opinion, but I understand in this way the changes that @fmilas proposed and @smoya introduced :)

[magicmatatjahu]

Additionally, one may wonder how to overwrite such an address if we want to have something like a channel but without the address? We can create a Channel Trait Object, just like we have a Message Trait Object and an Operation Trait Object, where address will be optional.

[char0n]

Maybe I don't understand the second paragraph (sorry!), but a channel key and an address are two completely different things in my opinion. I understand that the keys may be different, but this should be just "information" for the developer, not for a user reading the documentation. It's a bit like a server url - it may have a different key but url identifies its "uniqueness" and so it is with the channel address.

Don't worry; It's definitely me (not being clear). What I tried to say was this:

asyncapi: 2.3.0
info:
  title: reusable document fragment
  version: 1.0.0
components:
  channels:
    user/signup:
      description: channel description
channels:
  user/signup:
    $ref: #/components/channels/user~1signup

People can still use (and probably will) slashes in keys that represents reusable channels. I do understand the argument thought, that this proposal tries to avoid it as much as possible but is it worth it, given we have Components.channels which already solve this?

The last argument I have is about semantics:

Given definition 1:

channels:
  userSignedUp:
      address: user/signedup
      description: ...

And definition 2:

channels:
  userSignedUp:
      $ref: "./definition1.yaml/#/channels/userSignedUp"

When I change definition 1 to:

channels:
  userSignedUp:
      address: user/loggedout
      description: ...

Then resolution/dereferencing/bundling will still work But the semantics of what userSignedUp is, has changed And the channel now represents something completely different

The current status quo of {channel} being key of Channels Object guarantees that things fail fast if unique {channel} identifier changes, because if it changes it's probably an important change.

[char0n]

Hi guys,

I was thinking about this some more during the night, and looking at this change through the lens of different persona - as a tooling author (not definition writer) I actually like this change. This changes makes {channel} part of Channel Item Object which makes Channel Item Object self-contained. Currently when Channels Object is parsed into individual Channel Item Object elements, I have to attach to the {channel} (key under which the object is defined, which is very important information) as metadata to the particular Channel Item Object. With this change, maintaining the metadata will no longer be necessary.

And I want to thank you for giving me the opportunity to voice my opinions here (through two different personas - definitions author and tooling author) and sorry for hijacking this discussion a bit.

[smoya]

And I want to thank you for giving me the opportunity to voice my opinions here (through two different personas - definitions author and tooling author) and sorry for hijacking this discussion a bit.

Your opinion is super important on this (and also the opinion of anyone interested in the topic). I explicitly pinged you in the PR because I knew you had feedback to provide and most precisely, you were willing to provide it, which to be honest, in my mind, this is just awesome.

So please keep providing feedback around any PR you see because I think this is what it really matters on this community. In my case, whenever I open a new RFC, I'm looking actively for "Noes" rather than "Yes"; for "This is not gonna work" rather than "oh cool LGTM", you know what I mean :).

Thanks for your extremely valuable help here @char0n !

[magicmatatjahu]

Feedback (with other point of view) is very important and there are no apologies for it :) The more "no" votes the better because then we know from additional use cases.

Referring to the change itself: I wonder if when we have an address inside a Channel Item Object, shouldn't the .channels shape of the object be changed to a list instead of a dictionary?

asyncapi: 3.0.0
info: ...

channels: 
- $ref: '#/components/channels/someChannel'

Of course components.channels should still have the shape of an object to make references easier. WDYT? 🤔

[smoya]

Referring to the change itself: I wonder if when we have an address inside a Channel Item Object, shouldn't the .channels shape of the object be changed to a list instead of a dictionary?

Why wouldn't just look like servers? A map of unique ID => Channel Item Object.

[magicmatatjahu]

Why wouldn't just look like servers? A map of unique ID => Channel Item Object.

Currently channels look like servers as Map<string, Channel Item Object | Ref> however I wonder if it make sense if the address is no longer treated as a key but as an internal value. 🤔 Please treat it as an idea.

Of course that suggestion has problem with referencing when you want to make ref to .channels section like #/channels/someChannel.

Benefits:

• treat channels as channels available in the application, key is unnecessary and causes only additional boilerplate. Minuses:
• lack of support for references

[char0n]

Referring to the change itself: I wonder if when we have an address inside a Channel Item Object, shouldn't the .channels shape of the object be changed to a list instead of a dictionary?

This would IMHO make referencing even more difficult as people often change their specification and reorder stuff to make more sense or to look better. With using list semantics the indexes will change by reordering the internals of the Channels Object.

channels:
  - address: path/to/channel1
  - address: path/to/channel2

Pointer to channe1: JSON Pointer: #/channels/0 vs

channels:
  - address: path/to/channel2
  - address: path/to/channel1

Pointer to channe1: JSON Pointer: #/channels/1

[smoya]

I agree @char0n on this. Also I want to emphasize the fact that channels need a identifier. This is key, and one of the reasons behind this change. Read my comment here https://github.com/asyncapi/spec/issues/663#issuecomment-1016688987

[jessemenning]

This is a great discussion and it sounds like a good idea. Any chance we could have a brief real-time discussion of this, either at a v3 meeting or a dedicated tech discussion?

[jessemenning]

I would also be interested of the impact of this on #618 , especially the reference between operations and channels, which is proposed to not be a JSON ref. That seems suboptimal to me, from an understanding and tooling perspective.

Why isn't channel a $ref or JSON Pointer? To keep things simple. If you're referring to a channel here, it must be defined in the channels object. If we allow $ref here, it means it can be dereferenced and therefore the channel ID would be lost. To make things easier and more consistent, this field is simply a string with the name of the channel ID.

[damaru-inc]

I like this a lot, especially because it'll help reusability, i.e. breaking models down into reusable files.

[magicmatatjahu]

@jessemenning

I would also be interested of the impact of this on https://github.com/asyncapi/spec/issues/618 , especially the reference between operations and channels, which is proposed to not be a JSON ref. That seems suboptimal to me, from an understanding and tooling perspective.

Good point! I think that we have "little" problem here, however proposal proposed in this issue does not contradict the above sentence. If you want to use the operation, you must have a channel defined in the .channels field. We don't use JSON $ref here for a simple reason - the string tells us where to look (in the .channels field) and then we can be sure if the channel exists or not. If we were to use a reference like:

operations:
  sendUserSignedUpEvent:
    channel: 
      $ref: #/channels/someChannel
    action: send

then after dereferencing we have the channel object and not its identifier. To check if a given channel is in the context of the described documentation (in the .channels section) we would have to check deep equality between the object in .channels and the one defined by reference. Hopefully you understand that comparing string == string is much easier than object == object :)

[magicmatatjahu]

Of course, referencing can be a problem if we want to treat operations and channels in a registry fashion. Then we have to remember about those keys in the dictionary between files and I wonder if the operation and channel identifiers should be defined inside the object and not outside, but it's my little digression which we should discuss. cc @smoya @fmvilas

[smoya]

Of course, referencing can be a problem if we want to treat operations and channels in a registry fashion. Then we have to remember about those keys in the dictionary between files and I wonder if the operation and channel identifiers should be defined inside the object and not outside, but it's my little digression which we should discuss. cc @smoya @fmvilas

The reality is that this issue you talk about came to me several times when designing the Parser-API API.

Ideally, all objects would have an id field, which could be optional (if missing, the key of the map is used). The issue with that is that the issue when referencing with JSON Reference will be still there if the referenced object didn't declared the id field. Unless we use our own bundling tooling, which could then pick up somehow the ID from the key, I can't see how to make it happen easily.

Any ideas?

[magicmatatjahu]

@smoya

I see that problem. Probably we should create another discussion to clarify it.

I like that idea with id field. The only question is for which objects such an id would have to exist. I think the objects that should have something like this are server, channel, operation, message and maybe schema. The id itself could be inside info object, where we could define other "metadata" like contact, license, externalDocs etc (just like info for AsyncAPI document). And of course the id would be required:

channels:
  someChannel:
    info:
      id: ...
      externalDocs: ...
    ...
    
operations:
  someOperation:
    info:
      id: ...
      externalDocs: ...
    ...

A loose idea that needs to be discussed.

[jessemenning]

Of course, referencing can be a problem if we want to treat operations and channels in a registry fashion.

Agree and I think registries play a large part of AsyncAPI's future (paging @boyney123 and @dalelane ). I'm intrigued by @smoya 's comment about having id, but don't fully grasp the implications.

[mgiovann]

I'm coming here from #739 and I wanted to chime in and say that this proposal seems to support my use case. I want to represent that fact that my publish operation uses one URL (the URL of my message broker) and my subscribe operation uses another (the URL of my subscriber webhook). As @smoya has pointed out, the ability to define two different channels using the same channel address but separate servers would solve this.

[fmvilas]

Folks, on #806 I decided to define the channel field in the Operation Object as a Reference Object. This way:

operations:
  sendUserSignedUp:
    action: send
    channel:
      $ref: '#/channels/userSignedUp'

Please note that I'm not saying we should allow people to define channels inside the channel field. For instance, I'm not saying we should allow this:

operations:
  sendUserSignedUp:
    action: send
    channel:
      address: 'user/signedup'
      ...other channel stuff...

If we accept ONLY a $ref field there:

• IDEs will still recognize them, letting us better navigate documents with no additional effort required (all IDEs already have some kind of support for JSON Schema and $ref).
• We will be able to get the identifier before the dereferencing process.
• After dereferencing, we would have the whole object which is super convenient and we can annotate with the id obtained before dereferencing.
• We could even make a rule that the $ref value should start with #/channels/, otherwise it's invalid. Not sure this would be a good idea but it's a possibility.

[smoya]

IDEs will still recognize them, letting us better navigate documents with no additional effort required (all IDEs already have some kind of support for JSON Schema and $ref).

Do you mean some editors understand an AsyncAPI document (yaml) containing the $ref? I'm asking because I'm a bit confused., not saying this is not happening.

After dereferencing, we would have the whole object which is super convenient and we can annotate with the id obtained before dereferencing.

By annotating you mean adding a new field like id with the stored value (prior to dereferencing)?

[fmvilas]

Do you mean some editors understand an AsyncAPI document (yaml) containing the $ref? I'm asking because I'm a bit confused., not saying this is not happening.

IDEs with JSON Schema support understand $ref and some even let you navigate to the place the $ref is pointing to. They don't understand AsyncAPI but the plugins think it's a JSON Schema document and you can leverage some functionalities.

By annotating you mean adding a new field like id with the stored value (prior to dereferencing)?

Yes, that's exactly what I meant.

[derberg]

What do you mean by obtaining the ID before dereferencing? Like parsing ref address and simply getting last string after splitting with "/" or ?