ASB Concurrent Session FIFO

[jjcollinge]

Description

Fixes the concurrency handling of Azure Service Bus sessions to respect FIFO ordering.

Issue reference

Please reference the issue this PR will close: https://github.com/dapr/components-contrib/issues/3868

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

• [x] Code compiles correctly
• [x] Created/updated tests
• [ ] Extended the documentation
  • [ ] Created the dapr/docs PR: <insert PR link here>

Note: We expect contributors to open a corresponding documentation PR in the dapr/docs repository. As the implementer, you are the best person to document your work! Implementation PRs will not be merged until the documentation PR is opened and ready for review.

[jjcollinge]

Smoke test by running a dapr app with:

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: sessionpubsub
spec:
  type: pubsub.azure.servicebus.topics
  version: v1
  metadata:
  - name: connectionString
    value: "<REDACTED>"
  - name: maxConcurrentHandlers
    value: "10"
  - name: maxConcurrentSessions
    value: "10"
  - name: requireSessions
    value: "true"
    ...

Which demonstrated multiple sessions being concurrently processed whilst maintaining strict FIFO order and synchrnous processing for each session.

maxConcurrentHandlers is the total number of handlers that can be executed concurrently across all sessions.

maxConcurrentSessions is the total number of sessions that can be handled concurrently.

This means maxConcurrentSessions has precedence and if set to 1 effectively limits the maxConcurrentHandlers to 1 as well as only 1 message on a single topic can be processed at a time to ensure FIFO.

If maxConcurrentHandlers is lower than maxConcurrentSessions then you are limiting the throughput as sessions are waiting on each other.

[jjcollinge]

/ok-to-test

[dapr-bot]

Complete Build Matrix

The build status is currently not updated here. Please visit the action run below directly.

🔗 Link to Action run

Commit ref: 298ae264eb256e3dda2375fd61f91fd7fbed596a

[dapr-bot]

Components certification test

🔗 Link to Action run

Commit ref: 298ae264eb256e3dda2375fd61f91fd7fbed596a

❌ Some certification tests failed

These tests failed:

• bindings.azure.blobstorage
• bindings.azure.cosmosdb
• bindings.azure.eventhubs
• bindings.azure.servicebusqueues
• bindings.azure.storagequeues
• bindings.aws.s3
• bindings.cron
• bindings.dubbo
• bindings.zeebe.command
• bindings.zeebe.jobworker
• bindings.kafka
• bindings.kitex
• bindings.localstorage
• bindings.postgres
• bindings.rabbitmq
• bindings.redis
• configuration.postgres
• configuration.redis
• middleware.http.bearer
• middleware.http.ratelimit
• middleware.http.opa
• pubsub.aws.snssqs
• pubsub.gcp.pubsub
• pubsub.azure.eventhubs
• pubsub.azure.servicebus.topics
• pubsub.kafka
• pubsub.mqtt3
• pubsub.pulsar
• pubsub.rabbitmq
• secretstores.azure.keyvault
• secretstores.hashicorp.vault
• secretstores.local.env
• secretstores.local.file
• state.aws.dynamodb
• state.azure.blobstorage
• state.azure.cosmosdb
• state.azure.tablestorage
• state.cassandra
• state.cockroachdb.v1
• state.memcached
• state.mongodb
• state.mysql
• state.postgresql.v1
• state.postgresql.v2
• state.redis
• state.sqlite
• state.sqlserver
• state.sqlserver.v2
• state.gcp.firestore

Additionally, some tests did not report a status:

• bindings.azure.blobstorage
• bindings.azure.cosmosdb
• bindings.azure.eventhubs
• bindings.azure.servicebusqueues
• bindings.azure.storagequeues
• bindings.aws.s3
• bindings.cron
• bindings.dubbo
• bindings.zeebe.command
• bindings.zeebe.jobworker
• bindings.kafka
• bindings.kitex
• bindings.localstorage
• bindings.postgres
• bindings.rabbitmq
• bindings.redis
• configuration.postgres
• configuration.redis
• middleware.http.bearer
• middleware.http.ratelimit
• middleware.http.opa
• pubsub.aws.snssqs
• pubsub.gcp.pubsub
• pubsub.azure.eventhubs
• pubsub.azure.servicebus.topics
• pubsub.kafka
• pubsub.mqtt3
• pubsub.pulsar
• pubsub.rabbitmq
• secretstores.azure.keyvault
• secretstores.hashicorp.vault
• secretstores.local.env
• secretstores.local.file
• state.aws.dynamodb
• state.azure.blobstorage
• state.azure.cosmosdb
• state.azure.tablestorage
• state.cassandra
• state.cockroachdb.v1
• state.memcached
• state.mongodb
• state.mysql
• state.postgresql.v1
• state.postgresql.v2
• state.redis
• state.sqlite
• state.sqlserver
• state.sqlserver.v2
• state.gcp.firestore

[dapr-bot]

Components conformance test

🔗 Link to Action run

Commit ref: 298ae264eb256e3dda2375fd61f91fd7fbed596a

❌ Some conformance tests failed

These tests failed:

• bindings.azure.blobstorage
• bindings.azure.cosmosdb
• bindings.azure.eventgrid
• bindings.azure.eventhubs
• bindings.azure.servicebusqueues
• bindings.azure.storagequeues
• bindings.aws.s3.terraform
• bindings.cron
• bindings.http
• bindings.influx
• bindings.kafka-confluent
• bindings.kafka-wurstmeister
• bindings.kubemq
• bindings.mqtt3-emqx
• bindings.mqtt3-mosquitto
• bindings.mqtt3-vernemq
• bindings.postgresql.docker
• bindings.postgresql.azure
• bindings.rabbitmq
• bindings.redis.v6
• bindings.redis.v7
• configuration.postgresql.docker
• configuration.postgresql.azure
• configuration.redis.v6
• configuration.redis.v7
• crypto.azure.keyvault
• crypto.localstorage
• crypto.jwks
• lock.redis.v6
• lock.redis.v7
• pubsub.aws.snssqs.terraform
• pubsub.gcp.pubsub.terraform
• pubsub.azure.eventhubs
• pubsub.azure.servicebus.queues
• pubsub.azure.servicebus.topics
• pubsub.in-memory
• pubsub.jetstream
• pubsub.kafka-confluent
• pubsub.kafka-wurstmeister
• pubsub.kubemq
• pubsub.mqtt3-emqx
• pubsub.mqtt3-vernemq
• pubsub.pulsar
• pubsub.rabbitmq
• pubsub.redis.v6
• pubsub.solace
• secretstores.azure.keyvault.certificate
• secretstores.azure.keyvault.serviceprincipal
• secretstores.hashicorp.vault
• secretstores.kubernetes
• secretstores.local.env
• secretstores.local.file
• secretstores.aws.secretsmanager.terraform
• secretstores.aws.secretsmanager.docker
• state.aws.dynamodb.terraform
• state.azure.blobstorage.v2
• state.azure.blobstorage.v1
• state.azure.cosmosdb
• state.azure.sql
• state.azure.tablestorage.cosmosdb
• state.azure.tablestorage.storage
• state.cassandra
• state.cloudflare.workerskv
• state.cockroachdb.v1
• state.etcd.v1
• state.etcd.v2
• state.in-memory
• state.memcached
• state.mongodb
• state.mysql.mariadb
• state.mysql.mysql
• state.oracledatabase
• state.postgresql.v1.docker
• state.postgresql.v1.azure
• state.postgresql.v2.docker
• state.postgresql.v2.azure
• state.redis.v6
• state.redis.v7
• state.rethinkdb
• state.sqlite
• state.sqlserver
• state.sqlserver.v2
• state.sqlserver.docker
• state.sqlserver.v2.docker
• state.gcp.firestore.cloud

Additionally, some tests did not report a status:

• bindings.azure.blobstorage
• bindings.azure.cosmosdb
• bindings.azure.eventgrid
• bindings.azure.eventhubs
• bindings.azure.servicebusqueues
• bindings.azure.storagequeues
• bindings.aws.s3.terraform
• bindings.cron
• bindings.http
• bindings.influx
• bindings.kafka-confluent
• bindings.kafka-wurstmeister
• bindings.kubemq
• bindings.mqtt3-emqx
• bindings.mqtt3-mosquitto
• bindings.mqtt3-vernemq
• bindings.postgresql.docker
• bindings.postgresql.azure
• bindings.rabbitmq
• bindings.redis.v6
• bindings.redis.v7
• configuration.postgresql.docker
• configuration.postgresql.azure
• configuration.redis.v6
• configuration.redis.v7
• crypto.azure.keyvault
• crypto.localstorage
• crypto.jwks
• lock.redis.v6
• lock.redis.v7
• pubsub.aws.snssqs.terraform
• pubsub.gcp.pubsub.terraform
• pubsub.azure.eventhubs
• pubsub.azure.servicebus.queues
• pubsub.azure.servicebus.topics
• pubsub.in-memory
• pubsub.jetstream
• pubsub.kafka-confluent
• pubsub.kafka-wurstmeister
• pubsub.kubemq
• pubsub.mqtt3-emqx
• pubsub.mqtt3-vernemq
• pubsub.pulsar
• pubsub.rabbitmq
• pubsub.redis.v6
• pubsub.solace
• secretstores.azure.keyvault.certificate
• secretstores.azure.keyvault.serviceprincipal
• secretstores.hashicorp.vault
• secretstores.kubernetes
• secretstores.local.env
• secretstores.local.file
• secretstores.aws.secretsmanager.terraform
• secretstores.aws.secretsmanager.docker
• state.aws.dynamodb.terraform
• state.azure.blobstorage.v2
• state.azure.blobstorage.v1
• state.azure.cosmosdb
• state.azure.sql
• state.azure.tablestorage.cosmosdb
• state.azure.tablestorage.storage
• state.cassandra
• state.cloudflare.workerskv
• state.cockroachdb.v1
• state.etcd.v1
• state.etcd.v2
• state.in-memory
• state.memcached
• state.mongodb
• state.mysql.mariadb
• state.mysql.mysql
• state.oracledatabase
• state.postgresql.v1.docker
• state.postgresql.v1.azure
• state.postgresql.v2.docker
• state.postgresql.v2.azure
• state.redis.v6
• state.redis.v7
• state.rethinkdb
• state.sqlite
• state.sqlserver
• state.sqlserver.v2
• state.sqlserver.docker
• state.sqlserver.v2.docker
• state.gcp.firestore.cloud

[jjcollinge]

/ok-to-test

[dapr-bot]

Complete Build Matrix

The build status is currently not updated here. Please visit the action run below directly.

🔗 Link to Action run

Commit ref: de6d78aa5c669ab6e7f1bcb957cda2e3a8422ab8

[dapr-bot]

Components certification test

🔗 Link to Action run

Commit ref: de6d78aa5c669ab6e7f1bcb957cda2e3a8422ab8

❌ Some certification tests failed

These tests failed:

• pubsub.azure.servicebus.topics

Additionally, some tests did not report a status:

• pubsub.azure.servicebus.topics

[dapr-bot]

Components conformance test

🔗 Link to Action run

Commit ref: de6d78aa5c669ab6e7f1bcb957cda2e3a8422ab8

❌ Some conformance tests failed

These tests failed:

• pubsub.azure.servicebus.topics

Additionally, some tests did not report a status:

• pubsub.azure.servicebus.topics

[jjcollinge]

/ok-to-test

[dapr-bot]

Complete Build Matrix

The build status is currently not updated here. Please visit the action run below directly.

🔗 Link to Action run

Commit ref: ac25ba6f35a206e59e78024bd3a85cbe4904279d

[dapr-bot]

Components certification test

🔗 Link to Action run

Commit ref: ac25ba6f35a206e59e78024bd3a85cbe4904279d

❌ Some certification tests failed

These tests failed:

• bindings.azure.blobstorage
• bindings.azure.cosmosdb
• bindings.azure.eventhubs
• bindings.azure.servicebusqueues
• bindings.azure.storagequeues
• bindings.aws.s3
• bindings.cron
• bindings.dubbo
• bindings.zeebe.command
• bindings.zeebe.jobworker
• bindings.kafka
• bindings.kitex
• bindings.localstorage
• bindings.postgres
• bindings.rabbitmq
• bindings.redis
• configuration.postgres
• configuration.redis
• middleware.http.bearer
• middleware.http.ratelimit
• middleware.http.opa
• pubsub.aws.snssqs
• pubsub.gcp.pubsub
• pubsub.azure.eventhubs
• pubsub.azure.servicebus.topics
• pubsub.kafka
• pubsub.mqtt3
• pubsub.pulsar
• pubsub.rabbitmq
• secretstores.azure.keyvault
• secretstores.hashicorp.vault
• secretstores.local.env
• secretstores.local.file
• state.aws.dynamodb
• state.azure.blobstorage
• state.azure.cosmosdb
• state.azure.tablestorage
• state.cassandra
• state.cockroachdb.v1
• state.memcached
• state.mongodb
• state.mysql
• state.postgresql.v1
• state.postgresql.v2
• state.redis
• state.sqlite
• state.sqlserver
• state.sqlserver.v2
• state.gcp.firestore

Additionally, some tests did not report a status:

• bindings.azure.blobstorage
• bindings.azure.cosmosdb
• bindings.azure.eventhubs
• bindings.azure.servicebusqueues
• bindings.azure.storagequeues
• bindings.aws.s3
• bindings.cron
• bindings.dubbo
• bindings.zeebe.command
• bindings.zeebe.jobworker
• bindings.kafka
• bindings.kitex
• bindings.localstorage
• bindings.postgres
• bindings.rabbitmq
• bindings.redis
• configuration.postgres
• configuration.redis
• middleware.http.bearer
• middleware.http.ratelimit
• middleware.http.opa
• pubsub.aws.snssqs
• pubsub.gcp.pubsub
• pubsub.azure.eventhubs
• pubsub.azure.servicebus.topics
• pubsub.kafka
• pubsub.mqtt3
• pubsub.pulsar
• pubsub.rabbitmq
• secretstores.azure.keyvault
• secretstores.hashicorp.vault
• secretstores.local.env
• secretstores.local.file
• state.aws.dynamodb
• state.azure.blobstorage
• state.azure.cosmosdb
• state.azure.tablestorage
• state.cassandra
• state.cockroachdb.v1
• state.memcached
• state.mongodb
• state.mysql
• state.postgresql.v1
• state.postgresql.v2
• state.redis
• state.sqlite
• state.sqlserver
• state.sqlserver.v2
• state.gcp.firestore

[dapr-bot]

Components conformance test

🔗 Link to Action run

Commit ref: ac25ba6f35a206e59e78024bd3a85cbe4904279d

❌ Some conformance tests failed

These tests failed:

• bindings.azure.blobstorage
• bindings.azure.cosmosdb
• bindings.azure.eventgrid
• bindings.azure.eventhubs
• bindings.azure.servicebusqueues
• bindings.azure.storagequeues
• bindings.aws.s3.terraform
• bindings.cron
• bindings.http
• bindings.influx
• bindings.kafka-confluent
• bindings.kafka-wurstmeister
• bindings.kubemq
• bindings.mqtt3-emqx
• bindings.mqtt3-mosquitto
• bindings.mqtt3-vernemq
• bindings.postgresql.docker
• bindings.postgresql.azure
• bindings.rabbitmq
• bindings.redis.v6
• bindings.redis.v7
• configuration.postgresql.docker
• configuration.postgresql.azure
• configuration.redis.v6
• configuration.redis.v7
• crypto.azure.keyvault
• crypto.localstorage
• crypto.jwks
• lock.redis.v6
• lock.redis.v7
• pubsub.aws.snssqs.terraform
• pubsub.gcp.pubsub.terraform
• pubsub.azure.eventhubs
• pubsub.azure.servicebus.queues
• pubsub.azure.servicebus.topics
• pubsub.in-memory
• pubsub.jetstream
• pubsub.kafka-confluent
• pubsub.kafka-wurstmeister
• pubsub.kubemq
• pubsub.mqtt3-emqx
• pubsub.mqtt3-vernemq
• pubsub.pulsar
• pubsub.rabbitmq
• pubsub.redis.v6
• pubsub.solace
• secretstores.azure.keyvault.certificate
• secretstores.azure.keyvault.serviceprincipal
• secretstores.hashicorp.vault
• secretstores.kubernetes
• secretstores.local.env
• secretstores.local.file
• secretstores.aws.secretsmanager.terraform
• secretstores.aws.secretsmanager.docker
• state.aws.dynamodb.terraform
• state.azure.blobstorage.v2
• state.azure.blobstorage.v1
• state.azure.cosmosdb
• state.azure.sql
• state.azure.tablestorage.cosmosdb
• state.azure.tablestorage.storage
• state.cassandra
• state.cloudflare.workerskv
• state.cockroachdb.v1
• state.etcd.v1
• state.etcd.v2
• state.in-memory
• state.memcached
• state.mongodb
• state.mysql.mariadb
• state.mysql.mysql
• state.oracledatabase
• state.postgresql.v1.docker
• state.postgresql.v1.azure
• state.postgresql.v2.docker
• state.postgresql.v2.azure
• state.redis.v6
• state.redis.v7
• state.rethinkdb
• state.sqlite
• state.sqlserver
• state.sqlserver.v2
• state.sqlserver.docker
• state.sqlserver.v2.docker
• state.gcp.firestore.cloud

Additionally, some tests did not report a status:

• bindings.azure.blobstorage
• bindings.azure.cosmosdb
• bindings.azure.eventgrid
• bindings.azure.eventhubs
• bindings.azure.servicebusqueues
• bindings.azure.storagequeues
• bindings.aws.s3.terraform
• bindings.cron
• bindings.http
• bindings.influx
• bindings.kafka-confluent
• bindings.kafka-wurstmeister
• bindings.kubemq
• bindings.mqtt3-emqx
• bindings.mqtt3-mosquitto
• bindings.mqtt3-vernemq
• bindings.postgresql.docker
• bindings.postgresql.azure
• bindings.rabbitmq
• bindings.redis.v6
• bindings.redis.v7
• configuration.postgresql.docker
• configuration.postgresql.azure
• configuration.redis.v6
• configuration.redis.v7
• crypto.azure.keyvault
• crypto.localstorage
• crypto.jwks
• lock.redis.v6
• lock.redis.v7
• pubsub.aws.snssqs.terraform
• pubsub.gcp.pubsub.terraform
• pubsub.azure.eventhubs
• pubsub.azure.servicebus.queues
• pubsub.azure.servicebus.topics
• pubsub.in-memory
• pubsub.jetstream
• pubsub.kafka-confluent
• pubsub.kafka-wurstmeister
• pubsub.kubemq
• pubsub.mqtt3-emqx
• pubsub.mqtt3-vernemq
• pubsub.pulsar
• pubsub.rabbitmq
• pubsub.redis.v6
• pubsub.solace
• secretstores.azure.keyvault.certificate
• secretstores.azure.keyvault.serviceprincipal
• secretstores.hashicorp.vault
• secretstores.kubernetes
• secretstores.local.env
• secretstores.local.file
• secretstores.aws.secretsmanager.terraform
• secretstores.aws.secretsmanager.docker
• state.aws.dynamodb.terraform
• state.azure.blobstorage.v2
• state.azure.blobstorage.v1
• state.azure.cosmosdb
• state.azure.sql
• state.azure.tablestorage.cosmosdb
• state.azure.tablestorage.storage
• state.cassandra
• state.cloudflare.workerskv
• state.cockroachdb.v1
• state.etcd.v1
• state.etcd.v2
• state.in-memory
• state.memcached
• state.mongodb
• state.mysql.mariadb
• state.mysql.mysql
• state.oracledatabase
• state.postgresql.v1.docker
• state.postgresql.v1.azure
• state.postgresql.v2.docker
• state.postgresql.v2.azure
• state.redis.v6
• state.redis.v7
• state.rethinkdb
• state.sqlite
• state.sqlserver
• state.sqlserver.v2
• state.sqlserver.docker
• state.sqlserver.v2.docker
• state.gcp.firestore.cloud

[jjcollinge]

looks like the azure service bus topics certificate test passed? https://github.com/dapr/components-contrib/actions/runs/19430412140/job/55587725948

[sicoyle]

will look to merge this tomorrow unless I hear otherwise from ya @JoshVanL :)