docs
docs copied to clipboard
SNS/SQS boot and shutdown detailed description and edge cases
What content needs to be created or modified? The current AWS PubSub SNS/SQS shutdown behavior needs to be detailing the inherent limitation of the infrastructure when used by Dapr. Namely the fan-in behavior of multiple SNS topics that are transported into one SQS queue for consumption.
On boot scenarios:
- It is possible, especially under self provisioned infra (and typically
--disableEntityManagement
), that many topics are actively mapped to the consumer SQS, while the component is still gradually being called toSubscribe
by Dapr control plane for those topic - This results in messages arriving from SQS from topics the component has not yet been assigned to, causing error messages and potentially, if not well calibrated and without DLQ used, to data loss
On shutdown scenarios:
- Opposite to the boot, here the component can race and win the infra removal: no available message handlers would be mapped, while the infra still moves messages from SNS topics the component had already deregistered
Describe the solution you'd like We need to explain the users of the component about those issues and their origin and the limitation to length of efforts the component can take to making sure no misconsumption is performed
Where should the new material be placed? The sns/sqs component doc page
The associated pull request from dapr/dapr, dapr/components-contrib, or other Dapr code repos
https://github.com/dapr/components-contrib/pull/3174
Additional context The PR would at also resolve a potential issue of pre-termination of SQS message consumption (an edge case) so it is worth while mentioning that edge case and its resolution
/assign @amimimor