Add ability to support multiple Topics with Azure service bus

[yiliu-tc]

Currently, CAP only supports one topic in Azure service bus, but in real cases, application needs to be able to pub event to different Topics and sub to different Topics for consuming events.

Is it possible CAP could provide multiple Topics support for Azure service bus in next release cycle?

[yang-xiaodong]

Since my Azure subscription has expired, so I can't test it.

We don't consider further improvements for Azure Service Bus unless someone submits PR.

[mviegas]

I would be glad to participate in it. In fact, it is something I've been thinking about for a while now.

From the producer's perspective, how do we know which Topic should the Transport send a message?

• The publisher should be able to specify where it wants the message to published to. This could be specified in a Message Header. That way, it would be easy for other transports to implement the same capability (publish to multiple RabbitMQ exchanges, for example). Also, if the Header is not specified, it would send to the TopicPath property and ensure retrocompatibility.

From the consumer's perspective, it's a little trickier. Still haven't found a proper design that suits this use case. But I think that a more granular configuration would be needed - somewhat similar to what MassTransit does.

Didn't deep dive yet, these are my initial thoughts so far.

[mviegas]

Hey @yang-xiaodong! What do you think about this approach: https://github.com/mviegas/CAP/pull/1/files

It adds support to:

• Allow producers to specify via header where the message should be produced to;
• Allow custom subscribers, with a subscription-level configuration. For this, we had to create a ConsumerClientPool that holds all SubscriptionClientobjects, one for the default subscription, and another to each custom subscriber;
• Regarding custom subscribers, they:
  • Must specify the MessageType as well as the TopicPath;
  • Could specify some specific SubscriptionClient properties, in the as ReceiveMode and RetryPolicy;
  • Could specify a different ConnectionString if they are subscribing to another Service Bus namespace;

I was playing around with it today, but still have to dig a little deeper to understand if the design makes sense. Only then I would go deeper into more developments, tests and documentation.

Let me know what you think.

[yiliu-tc]

Viegas, thanks so much for your changes.

Do you have brief example of how to use handy? I can help to verify the changes as I have Env setup here.

Kind Regards,

Yi

[yang-xiaodong]

Hello @mviegas, it is Chinese New Year recently, I will have time to check after the festival, about 10 days later

[yiliu-tc]

If need anyone with azure subscription to test and verify, I can help

[mviegas]

Just got back to these developments and decided to follow a phased approach.

First, will add the possibility of publishing to more than one Topic. This is WIP here if anyone wants to check it. Starting with publishing as it is simpler than subscribing when regarding to multiple topics.

Will do some unit/integration tests to only then submit a PR to the official repo here.

[jonekdahl]

We would be interested in using this feature since we also have use cases where a single Service Bus topic is not enough.

[jonekdahl]

@mviegas I see that #1139 was closed because of lack of time. May I ask if you are still planning to work on this when you have more time?

[mviegas]

Hi @jonekdahl !

Yes, it is a feature I'd like to add but haven't gotten much time to get back to it yet. But if you want to collaborate with it, I'm happy in participating in design discussions and reviewing a PR!

[jonekdahl]

Hey @mviegas, I rebased your changes from #1139 on the master branch and opened a draft PR (#1283).