argo-events icon indicating copy to clipboard operation
argo-events copied to clipboard

The AzureEventhubSource should receive events with all consumer groups along with the default one.

Open beinghumane opened this issue 1 year ago • 3 comments

Describe the bug Azure EventHub source is listening with the event hub fqdn and eventhub/topic name. It doesn't seem to have a configuration field to listen as a consumer group. This limits scaling as there can be only 5 concurrent readers per partition/consumer and it's listening only as the default consumer group.

To Reproduce Set up a consumer group in the Azure Eventhub. Try to receive events with that consumer group. No such option seems to be available in the AzureEventHubSource config. It seems to receive events with the default consumer group. Did a quick code dive and call stack goes down until:

newReceiver() => // "newReceiver creates a new Service Bus message listener given an AMQP client and an entity path

Seems to use the default consumer group.

Expected behavior Ability to receive events with all the consumer groups. With EventHub std subscription, there can be up to 20 consumer groups, but unable to leverage its use without having the option to configure consumer group in the event-source and consume events. Please do suggest any alternate approaches to sort this out as well.

Environment (please complete the following information):

  • Argo: [3.4.4]
  • Argo Events: [v1.7.3]

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

beinghumane avatar Dec 08 '22 06:12 beinghumane