[BUG] Azure Service Bus: 'Listen' claim(s) are required to perform this operation

[ahrytsiuk]

Describe the bug We use Spring Cloud Stream binder for Azure Service Bus.

<dependency>
    <groupId>com.azure.spring</groupId>
    <artifactId>spring-cloud-azure-stream-binder-servicebus</artifactId>
</dependency>

After we upgraded spring-cloud-azure-dependencies BOM

<dependency>
    <groupId>com.azure.spring</groupId>
    <artifactId>spring-cloud-azure-dependencies</artifactId>
    <version>${spring-cloud-azure.version}</version>
    <type>pom</type>
    <scope>import</scope>
</dependency>

from from 5.17.1 to 5.18.0 we start seeing error in the logs (no code changes were made).

Exception or Stack Trace

{"@timestamp":"2024-12-12T10:26:40.293704942Z","@version":"1","message":"{\"az.sdk.message\":\"onLinkRemoteClose\",\"connectionId\":\"MF_930e45_1733999191076\",\"errorCondition\":\"amqp:unauthorized-access\",\"errorDescription\":\"Unauthorized access. 'Listen' claim(s) are required to perform this operation. Resource: 'sb://sb-r....servicebus.windows.net/.../subscriptions/...'. TrackingId:884f5ba335064977a02d0084d8b5f866_G22, SystemTracker:gateway10, Timestamp:2024-12-12T10:26:39\",\"linkName\":\"....\",\"entityPath\":\"...\"}","logger_name":"com.azure.core.amqp.implementation.handler.ReceiveLinkHandler2","thread_name":"reactor-executor-2","level":"INFO","level_value":20000}

{"@timestamp":"2024-12-12T10:26:40.362354504Z","@version":"1","message":"{\"az.sdk.message\":\"Receiver emitted terminal error.\",\"exception\":\"Unauthorized access. 'Listen' claim(s) are required to perform this operation. Resource: 'sb://....servicebus.windows.net/.../subscriptions/...'. TrackingId:884f5ba335064977a02d0084d8b5f866_G22, SystemTracker:gateway10, Timestamp:2024-12-12T10:26:39, errorContext[NAMESPACE: sb-rndce-dev-511.servicebus.windows.net. ERROR CONTEXT: N/A, PATH: .../subscriptions/email-group, REFERENCE_ID: ..., LINK_CREDIT: 0]\",\"messageFlux\":\"mf_fbfb3c_1733999191075\",\"connectionId\":\"MF_930e45_1733999191076\",\"linkName\":\"..../subscriptions/....\",\"entityPath\":\"...\"}","logger_name":"com.azure.core.amqp.implementation.MessageFlux","thread_name":"receiverPump-3","level":"WARN","level_value":30000,"stack_trace":"com.azure.core.amqp.exception.AmqpException: Unauthorized access. 'Listen' claim(s) are required to perform this operation. Resource: 'sb://sb....windows.net/...'. TrackingId:884f5ba335064977a02d0084d8b5f866_G22, SystemTracker:gateway10, Timestamp:2024-12-12T10:26:39, errorContext[NAMESPACE: sb-rndce-dev-511.servicebus.windows.net. ERROR CONTEXT: N/A, PATH: .../subscriptions/email-group, REFERENCE_ID: ..., LINK_CREDIT: 0]\n\tat com.azure.core.amqp.implementation.ExceptionUtil.toException(ExceptionUtil.java:90)\n\tat com.azure.core.amqp.implementation.handler.LinkHandler.handleRemoteLinkClosed(LinkHandler.java:120)\n\tat com.azure.core.amqp.implementation.handler.LinkHandler.onLinkRemoteClose(LinkHandler.java:63)\n\tat com.azure.core.amqp.implementation.handler.ReceiveLinkHandler2.onLinkRemoteClose(ReceiveLinkHandler2.java:176)\n\tat org.apache.qpid.proton.engine.BaseHandler.handle(BaseHandler.java:176)\n\tat org.apache.qpid.proton.engine.impl.EventImpl.dispatch(EventImpl.java:108)\n\tat org.apache.qpid.proton.reactor.impl.ReactorImpl.dispatch(ReactorImpl.java:324)\n\tat org.apache.qpid.proton.reactor.impl.ReactorImpl.process(ReactorImpl.java:291)\n\tat com.azure.core.amqp.implementation.ReactorExecutor.run(ReactorExecutor.java:91)\n\tat reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68)\n\tat reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28)\n\tat java.base/java.util.concurrent.FutureTask.run(Unknown Source)\n\tat java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)\n\tat java.base/java.lang.Thread.run(Unknown Source)\n"}

Expected behavior Service should connect to ASB without error.

Unfortunately I wasn't able to find anything in the release notes what could cause this error.

Could you please tell if we need to add some configuration after upgrading?

Edits

• Updated affected version from 5.14.0 to 5.17.1

[github-actions[bot]]

@anuchandy @conniey @lmolkova

[github-actions[bot]]

Thank you for your feedback. Tagging and routing to the team member best able to assist.

[yunbozhang-msft]

Hi team,

We also encounter this issue after upgrade to 5.18,

reactor.core.Exceptions$ErrorCallbackNotImplemented: com.azure.core.amqp.exception.AmqpException: Unauthorized access. 'Send' claim(s) are required to perform this operation.

[conniey]

If you bump the version to 5.19.0, do you still get this error? If so, could you provide a repro where this occurs?

[github-actions[bot]]

Hi @ahrytsiuk. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

[ahrytsiuk]

Hello @conniey, yes issue is still reproducible with 5.19.0. Let me prepare minimal reproducible example.

[ahrytsiuk]

@conniey here is a sample - https://github.com/ahrytsiuk/playground-asb

If you update spring-cloud-azure in the pom.xml from 5.14.0 to 5.19.0 you should see:

com.azure.messaging.servicebus.ServiceBusException: Unauthorized access. 'Listen' claim(s) are required to perform this operation. Resource: 'sb://sb-<...>.servicebus.windows.net/echo/subscriptions/test'. TrackingId:5316749a1e924a758f9aa370a70df3a5_G16, SystemTracker:gateway10, Timestamp:2025-01-13T11:05:00, errorContext[NAMESPACE: <...>.servicebus.windows.net. ERROR CONTEXT: N/A, PATH: echo/subscriptions/test, REFERENCE_ID: echo/subscriptions/test_d3ec73_1736766299169, LINK_CREDIT: 0]

[anuchandy]

Hello @ahrytsiuk, thanks for the sample!. Do you see this issue if you use a brand-new Service Principal (steps)?

Since the app both listens to and send message to the topic/subscription, the service principal requires Azure Service Bus Data Receiver and Azure Service Bus Data Sender roles

Just quickly tried the sample you prepared with <spring-cloud-azure.version>5.19.0</spring-cloud-azure.version> and I see both sender and listener functioning

2025-01-22T14:11:10.803-08:00  INFO 17940 --- [playground-asb] [nio-8080-exec-1] o.a.k.c.t.i.KafkaMetricsCollector        : initializing Kafka metrics collector
2025-01-22T14:11:10.823-08:00  INFO 17940 --- [playground-asb] [nio-8080-exec-1] o.a.kafka.common.utils.AppInfoParser     : Kafka version: 3.7.1
2025-01-22T14:11:10.823-08:00  INFO 17940 --- [playground-asb] [nio-8080-exec-1] o.a.kafka.common.utils.AppInfoParser     : Kafka commitId: e2494e6ffb89f828
2025-01-22T14:11:10.823-08:00  INFO 17940 --- [playground-asb] [nio-8080-exec-1] o.a.kafka.common.utils.AppInfoParser     : Kafka startTimeMs: 1737583870823
2025-01-22T14:11:10.845-08:00  INFO 17940 --- [playground-asb] [ad | producer-1] org.apache.kafka.clients.Metadata        : [Producer clientId=producer-1] Cluster ID: ..
2025-01-22T14:11:10.853-08:00  INFO 17940 --- [playground-asb] [nio-8080-exec-1] o.s.c.s.m.DirectWithAttributesChannel    : Channel 'playground-asb.echoListener-out-0' has 1 subscriber(s).
2025-01-22T14:11:10.854-08:00  INFO 17940 --- [playground-asb] [nio-8080-exec-1] o.s.c.s.binder.kafka.KafkaBinderMetrics  : Try to shutdown the old scheduler with 1 threads
2025-01-22T14:11:10.950-08:00  INFO 17940 --- [playground-asb] [container-0-C-1] c.g.ahrytsiuk.playground.EchoListener    : New message received: 'test'
2025-01-22T14:11:30.408-08:00  INFO 17940 --- [playground-asb] [nio-8080-exec-6] c.g.ahrytsiuk.playground.EchoController  : /echo called with param=>test
2025-01-22T14:11:30.417-08:00  INFO 17940 --- [playground-asb] [container-0-C-1] c.g.ahrytsiuk.playground.EchoListener    : New message received: 'test'

[ahrytsiuk]

Hello @anuchandy, how application configuration should change if I want to use Service Principal.

What could be the reason of the failure which was reported? Application worked before with this config:

Is there any way to fallback to previous behaviour? Maybe using config properties?

[anuchandy]

Hi @ahrytsiuk, I thought you're already using Service Principal based on the application.yml file in the sample you shared. Your application.yml already had the following section, and when running the sample, I provided values for those properties -

azure:
    credential:
    client-id: ${AZURE_CLIENT_ID}
    client-secret: ${AZURE_CLIENT_SECRET}
    profile:
    tenant-id: ${AZURE_TENANT_ID}
    subscription-id: ${AZURE_SUBSCRIPTION_ID}
    servicebus:
    namespace: ${AZURE_SERVICEBUS_NAMESPACE}
    resource:
        resource-group: ${AZURE_SERVICEBUS_RESOURCE_GROUP}

To assign Listen (Receiver) and Send roles to the Service Principal, I used the portal. The Access Control (IAM) section on the Service Bus topic portal page allows you to add the Azure Service Bus Data Receiver and Azure Service Bus Data Sender roles to the Service Principal instance.

If you want to use the connection string instead (as appear in the screen shot you shared), there is connection-string configuration property that you should be able to use. Here is the configuration properties documented: Azure Service Bus configuration properties - Java on Azure | Microsoft Learn

[ahrytsiuk]

@anuchandy sorry for the confusion. I'm not very familiar with terminology. It seems we using Service Principal. Properties from the sample I shared represent what is used for our application. We don't use connection string (aka spring.cloud.azure.servicebus.connection-string).

With screenshot I shared above I wanted to make a point that we already have Manage, Send and Listed claims (at least this is what I see, but I might be wrong). And before library update application was able to produce/consumer messages to/from Service Bus.

I'll double check with our Ops people what exact Claims are configured.

[anuchandy]

Hello @ahrytsiuk, no worries. The screenshot you shared earlier is the section of the page where we configure connection-string for Service Bus. If you select that checkbox, you should see them, the claims (Manage) in that page are the claims assigned specifically to the connection-string, if you were to use connection-string to authenticate.

When using a Service Principal (SP) to authenticate, the roles (Azure Service Bus Data Receiver and Azure Service Bus Data Sender) are assigned to SP under the IAM section (the third item in the left panel).

[ahrytsiuk]

Hello @anuchandy, I was debugging this issue more carefully, and found what is causing this error.

Extra check in the AbstractAzureServiceClientBuilderFactory which added here https://github.com/Azure/azure-sdk-for-java/commit/95321606db9dfe8364ad068fd35c992d4d3f6735#diff-f5b8ebf17d60689ce3e1d2c974329cddc2c03408d613effa1018d8c42b03746cR239 changed auth flow:

if (credentialConfigured) {
    return;
}

Before this change (prior version 5.17.1 of spring-cloud-azure-core) I can see in the logs such entries:

...
Will configure the credential of type ClientSecretCredential for ServiceBusClientBuilder.
Connection string configured for class ServiceBusClientBuilder.
...
Will configure the credential of type ClientSecretCredential for ServiceBusClientBuilder.
Connection string configured for class ServiceBusClientBuilder.

Note that it appears 2 times.

So application used connection string generated by ServiceBusArmConnectionStringProvider based on the parameters from the config

Endpoint=sb://sb-*.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=***

After this change - starting from 5.18.0 this behaviour changed. Is it possible to restore it? Or make it under feature flag?

Also I can't find anything related this unfortunate breaking change in the release notes.

P.S.: meanwhile I'll try to check if it's feasible for us to change env config. Which for us is the least favorable option.

[ahrytsiuk]

Hello @anuchandy, adding roles explicitly Azure Service Bus Data Sender and Azure Service Bus Data Receiver did the trick. Thank you for pointing in the right direction?

So this issue is not blocking us anymore, but could you check if changes introduced here were intentional. And it was expected for auth flow to change?

[anuchandy]

Hello @ahrytsiuk, glad to hear that you're unblocked with IAM route.

Thanks for sharing the observation regarding Spring AbstractAzureServiceClientBuilderFactory. Let me loop in Spring binder experts to review this finding. Hi @saragluna, could you please take a look?

[rujche]

Hi, @ahrytsiuk

I can not reproduce the problem using your example project: https://github.com/ahrytsiuk/playground-asb. Here is the screenshot:

I already assigned required role to the service principal. How did you make sure the service principal have authority to send & receive message before assign required role to the service principal? You said that you didn't use connection string.