spring-cloud-aws
spring-cloud-aws copied to clipboard
[SQS] Priority for queues while listening
Is your feature request related to a problem? Please describe.
I would like to define a priority in @SqsListener so that messages from certain high priority queues are given precedence over normal queues.
Describe the solution you'd like
The most basic implementation could change SimpleMessageListenerContainer to accept 2 taskExecutors - one for high priority and another for normal priority. SimpleMessageListenerContainer can then use appropriate taskExecutor based on priority defined in @SqsListener annotation.
A more advanced implementation would be like:
- Maintain bounded queues for each priority
- All
AsynchronousMessageListenerthreads push messages to appropriate bounded queues. - A separate thread polls messages from bounded queues based on priority and submits task to
taskExecutorfor handling messages
Or we could use streams:
- Build a stream for each queue
- Combine streams based on priority
- Register callback on combined stream that submits task to
taskExecutorfor handling messages
Describe alternatives you've considered Can't think of any other alternatives
Additional context If we do advanced implementation or use streams, we also fix the current issue with SimpleMessageListenerContainer that polls messages and waits for all of them to be handled before polling for new messages. That delays processing of a queue is a single message takes too long to process. See https://github.com/Mercateo/sqs-utils