Resource aware SimpleMessageListenerContainer [AMQP-420]

[spring-operator]

Wilfredo Sellado opened AMQP-420 and commented

Our application can process 400K XML files of 2MB in size per hour setting the 80 threads in SimpleMessageListernerContainer as the max. However, there are times that a bursts of at leat 15MB XML need to processed. Having this scenario in 80 threads 15MB is 1,200 MB in total. The max heap specified for the application is only 1.5G which obviously out of memory error can happen.

SimpleMessageListernerContainer logic on when to add or stop a consumer is just based on where the thread is still active or not. I would like to propose if resouce availability can be added on this logic. Currently, I just took over the class just to put my fix. What I did is create an interface ResouceEvaluator with available() method. And this ResourceEvaluator can be set externally by passing a specific implementation. And in the method, AsyncMessageProcessingConsumer.run(), I have added the call to this ResouceEvaluator.available(). If returns true, means active consumer can continue receiving message and suggest to add new ones. If false, means the current active consumer should be considered to be stopped immediately bypassing the idle tigger value.

Regarding my specific implementation of the ResourceEvaluator, i just check the free availble memory and cpu usage based on the com.sun.management.OperatingSystemMXBean(I am using oracle JRE).

Please let me know if this makes sense.

---

Affects: 1.3.5