spring-cloud-aws
spring-cloud-aws copied to clipboard
[SQS] Priority for queues while listening
Is your feature request related to a problem? Please describe.
I would like to define a priority in @SqsListener
so that messages from certain high priority queues are given precedence over normal queues.
Describe the solution you'd like
The most basic implementation could change SimpleMessageListenerContainer
to accept 2 taskExecutor
s - one for high priority and another for normal priority. SimpleMessageListenerContainer
can then use appropriate taskExecutor
based on priority defined in @SqsListener
annotation.
A more advanced implementation would be like:
- Maintain bounded queues for each priority
- All
AsynchronousMessageListener
threads push messages to appropriate bounded queues. - A separate thread polls messages from bounded queues based on priority and submits task to
taskExecutor
for handling messages
Or we could use streams:
- Build a stream for each queue
- Combine streams based on priority
- Register callback on combined stream that submits task to
taskExecutor
for handling messages
Describe alternatives you've considered Can't think of any other alternatives
Additional context If we do advanced implementation or use streams, we also fix the current issue with SimpleMessageListenerContainer that polls messages and waits for all of them to be handled before polling for new messages. That delays processing of a queue is a single message takes too long to process. See https://github.com/Mercateo/sqs-utils