JMS Listener threads block one another after deleteMessage failure

[justinfalk]

After a couple of days and thousands of successfully processed messages our JMS message listener encountered a server-side AWS error. While attempting to recover it blocked all of the threads which required the application to be restarted.

2018-07-26 10:06:49,172 WARN  -> (handler-6) [DefaultMessageListenerContainer] Setup of JMS message listener invoker failed for destination 'MY_EXAMPLE_QUEUE.fifo' - trying to recover. Cause: AmazonServiceException: deleteMessage. RequestId: e4796bb3-1242-5fc0-9277-3860f7f7dbf8
HTTPStatusCode: 500 AmazonErrorCode: InternalError
2018-07-26 10:06:49,172 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:06:49,172 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:06:55,309 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:06:55,310 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:00,833 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:00,833 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:18,390 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:18,390 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:18,411 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:18,411 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:22,651 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:22,651 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:28,260 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:28,260 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:44,331 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:44,331 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:50,191 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:50,192 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:50,211 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:50,211 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:50,234 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:50,240 INFO  -> (handler-6) [DefaultMessageListenerContainer] Successfully refreshed JMS Connection

[justinfalk]

Appears similar to #46

[rkass]

@justinfalk Were you ever able to get more insight on this? We're experiencing the exact same log stream without the access denied errors of #46

[stevenzyang]

This has been fixed as of https://github.com/awslabs/amazon-sqs-java-messaging-lib/pull/41