spring-cloud-aws
spring-cloud-aws copied to clipboard
Why there is only one polling thread? Can't we increase the number?
Can you please add the implementation to have multiple threads polling one queue. As of today only one thread polls right.
Here is the problem, During a busy hour ,As per metric from cloud watch average number of messages received per SQS poll is 4. And each poll is literally taking 750 ms. If i calculate this way in one minute i will be only able to get 320 messages. This is limiting performance of my application.
If i have multiple threads polling and if i can control that through a flag it will be a lot of help to many people.
Even If i pass same queueURL twice to @sqsListener it will not work. because in the start method of simpleMessageListener we have a check to see if queue is already started. We need some way to have multiple polling threads.
protected void startQueue(String queueName, QueueAttributes queueAttributes) {
if (this.runningStateByQueue.containsKey(queueName) && this.runningStateByQueue.get(queueName)) {
return;
}
this.runningStateByQueue.put(queueName, true);
Future<?> future = getTaskExecutor().submit(new AsynchronousMessageListener(queueName, queueAttributes));
this.scheduledFutureByQueue.put(queueName, future);
}
Is there any update on this? WE are very interested in this as well
SQS integration design will be revisited for 3.0. There are so many issues with current implementation that it needs solid amount of work to get it straight.
@maciejwalkowiak is there an estimate somewhere for the 3.0 release?
@maciejwalkowiak Not sure if it's worth opening another ticket, but has there ever been consideration of allowing the coder to specify different TaskExecutors per-listener? My understanding is that if you set the SimpleMessageListenerContainer's setTaskExecutor method to use a thread-pool, that pool will be shared across all listeners. In my case it would be nice if you could customize a particular listener or listeners to have more/less threads or set different keep-alive times.
We won't have time to implement it in 2.x. SQS needs solid rewrite which will happen in 3.0. If you come up with an elegant implementation for 2.x I am happy to review and merge your PR @rhanton.