Can I dynamically modify SQS queueName/url at runtime ?

[naveen-rd]

trafficstars

This is a great library that I came across (at medium.com) when looking for dynamic changes at runtime for my SQS based spring-boot microservice.

    @QueueListener("${foo.sqs.queue.url}")
    public void processMessage(@Payload final String payload) {
        // process the message payload here
    }

I am using QueueListener which at the app startup loads foo queue url just fine. But while the app is up and running processing events from the queue, if I want to dynamically change the queue to a different queue, say foo-test url, does your library support it? Could you please provide a sample of the same ?

Background: I am trying to run a test queue part of my Blue-Green deployment and the deployment would start with the test queue. Once messages from the test queue are processed and the test is successful in the passive stack, I want to dynamically change the queue to the production queue.

Please advise on how to achieve this. If there are alternative approaches, let me know. Thanks in advance!

[JaidenAshmore]

Hey, thanks for the interest!

So short answer, there is not a way with the way you are describing but there should be a way we can do this.

What we need to do is:

• Make sure on startup no queues are started
• We enable your canary queue listener at this point
• You do your checks to make sure it is healthy
• We disable the canary queue listener
• We enable the prod queue listener

How you can do this?

We can implement our own version of this interface: https://github.com/JaidenAshmore/java-dynamic-sqs-listener/blob/df9e7a64d2c806d8c084cde2e5bacc9524243600/spring/spring-core/src/main/java/com/jashmore/sqs/spring/container/DefaultMessageListenerContainerCoordinatorProperties.java#L11

For example your code could look like:

        @Bean
        public DefaultMessageListenerContainerCoordinatorProperties defaultMessageListenerContainerCoordinatorProperties() {
            return StaticDefaultMessageListenerContainerCoordinatorProperties.builder()
                          .isAutoStartContainersEnabled(false)
                          .build();
        }

We add two queue listeners onto the same method, both of these will not be listening at the start

   @QueueListener(value = "${foo.sqs.queue.canary.url}", id="canary-queue")
   @QueueListener(value = "${foo.sqs.queue.url}", id="prod-queue")
    public void processMessage(@Payload final String payload) {
        // process the message payload here
    }

Somewhere in your code you need to enable the canary queue:

@Service
public class MyService {
    private final MessageListenerContainerCoordinator messageListenerContainerCoordinator;

   @Autowired
   public MyService( final MessageListenerContainerCoordinator messageListenerContainerCoordinator) {
           this.messageListenerContainerCoordinator = messageListenerContainerCoordinator;
    }

    // this could be listening to the application start or an event listener, etc
    public void startCanaryQueue() {
           messageListenerContainerCoordinator.startContainer("canary-queue");
    }

    // this could be listening to the application start or an event listener, etc
    public void canarySuccess() {
           messageListenerContainerCoordinator.stopContainer("canary-queue");
           messageListenerContainerCoordinator.startContainer("prod-queue");
    }
}

The negative to the above approach is that all queues will be disabled at the start so if you had other queues you wanted to be enabled but have the prod one disabled you could do something like:

public void enableNonProdQueuesForStartup() {
     return messageListenerContainerCoordinator.getContainers().stream()
             .map(MessageListenerContainer::getidentifier)
             .filter(id -> !id.equals("prod-queue"))
             .forEach(messageListenerContainerCoordinator::startContainer);
}

let me know if that works!

[naveen-rd]

Hey @JaidenAshmore , Thanks for sharing this approach. I will try this out and update here.

[naveen-rd]

I tried this approach and looks like, using more than QueueListener in a method is not allowed using your library.

   @QueueListener(value = "${foo.sqs.queue.canary.url}", id="canary-queue")
   @QueueListener(value = "${bar.sqs.queue.url}", id="prod-queue")
    public void processMessage(@Payload final String payload) {
        // process the message payload here
    }

Duplicate annotation. The declaration of 'com.jashmore.sqs.spring.container.basic.QueueListener' does not have a valid java.lang.annotation.Repeatable annotation

[JaidenAshmore]

ah yeah of course, just do two public functions then

   @QueueListener(value = "${foo.sqs.queue.canary.url}", id="canary-queue")
    public void processMessageCanary(@Payload final String payload) {
        processMessage(payload);
    }

   @QueueListener(value = "${bar.sqs.queue.url}", id="prod-queue")
   public void processMessageProd(@Payload final String payload) {
        processMessage(payload);
    }
   private void processMessage(@Payload final String payload) {
        // process the message payload here
    }