Separate request/response queues?

[MNGoldenEagle]

Greetings!

I'm currently investigating how to implement an asynchronous request-response pattern using SQS/SNS, but with a few added wrinkles into the mix. When the API receives the initial request we want to forward it on to a separate service via SQS, perform some additional processing over a number of services (effectively lambdas, but not actual Lambdas) using SNS/SQS, and then send the response back to the API service via SQS. If this process takes longer than the API service is willing to wait (around 10 seconds), then we want it to fail-over to a separate queue so a different processor can pick it up and run with it.

This is a pretty complex use case, I know, and while I had initially hoped that this would work with this library, I'm struggling to figure out how to make this work. So I thought I'd run some of this by you guys and a) make sure my assumptions check out, and b) gather any suggestions on how I should approach this.

The biggest thing I've noticed is that the vast majority of the code assumes that the requester and responder are utilizing the same queue. I might be completely off track here, but I don't see any way to specify in the API two separate queues.

The second thing I've noticed is that the VirtualQueueClient doesn't seem to support multiple JVMs on the same queue, unless I'm really missing something. Specifically, there doesn't appear to be any logic for handling virtual queues created by other JVMs or processes; it looks like it just treats them as orphaned, since as far as I can tell it doesn't attempt to synchronize the virtual queues across hosts.

I've been working on drafting up a proof-of-concept that demonstrates this use case, but I'm starting to think I might need to modify the client code itself in order to support the concept of dual queues. That would at least let me use the original RequesterClient and ResponderClient libraries.

Is this a use case you guys support/are interested in supporting? I can continue investigating into this if so.

[adam-aws]

Hi there, you are correct in your assumptions. The requester and responder use the same queue, when the requester sends a message, the client creates a virtual queue and attaches the URL of this virtual queue to the message as an internal message attribute, so that the server knows where to send the response.

At the moment we have no plans on supporting such a feature, allowing the use of separate queues.

I can leave this issue open for the moment in case there is more interest in this.