Custom exchanges for REQ/REP sockets

[zachgoldstein]

I would like to create a REQ or REP socket on a particular exchange. I think rabbit.js just uses the default exchanges for these sockets. Not sure how to achieve this and I believe it's exposed in amqlib. Is it as simple as adding another param to ReqSocket.connect and using assertExchange?

[squaremo]

Is it as simple as adding another param to ReqSocket.connect

Do you want replies to go back via an exchange as well, or just the requests?

[zachgoldstein]

Yup, replies as well. Ideally like control over the exchange via the connect for all sockets really. It's not clear to me what socket.connect does off the bat. It either creates a queue or an exchange based on the socket type. Best would be an options object I could pass in there with queueName, exchangeName, etc.

This is what I've done to patch it for now:

self.dummySoc.constructor.prototype.connect = function(queueName, exchangeName, callback) {
    var socket = this;
    var ch = this.ch;

    if (this.consumers[queueName]) {
        delay(callback);
        return;
    }
    ch.assertExchange(exchangeName, 
        socket.options.routing || 'direct',
        { durable: socket.options.persistent }
    ).then(function() {
        return ch.assertQueue(queueName, {
            durable: socket.options.persistent
        });
    }).then(function(ok) {
        return ch.consume(queueName, function(msg) {
            if (msg !== null) {
                socket.requests.push(msg);
                socket.push(msg.content);
            } else socket.push(null);
        }, {
            noAck: false
        }).then(function(ok) {
            socket.consumers[queueName] = ok.consumerTag;
        });
    }).then(callback || ignore);
};

[squaremo]

The complications are:

1. The return path must be encoded in each request. The conventional way to do this is to put a queue name in the 'replyTo' header. Having some other encoding (to include an exchange name) would break this convention, and compatibility with any libraries that follow it. There's also limited space -- the header has up to 255 chars, and both queue names and exchange names are up to 255 chars.
2. Having routing in the middle breaks the REQ/REP contract. Requests can be dropped if there's no-one currently listening, or go to more than one replier, and the requesting socket won't know whether or not to expect an answer, or more than one answer. See the discussion under #57.

[squaremo]

Another (weaker) objection is that the point of rabbit.js is to not have to think about exchanges and queues.

What is your motivation for sending via an exchange?

[zachgoldstein]

Basically if we've got a variety of different services, each one doing rpc, we can do something like namespacing via the exchanges and not have to worry about something going really wrong and binding to another service's queues. Point 1. makes total sense. Bit confused about point 2. It still goes to the default exchange, what about that exchange prevents those problems?

[zachgoldstein]

Also, setting a different exchange than the default allows us to set durability and auto-delete and all that. Those choices can be a bit application-specific and having access to that flexibility via the socket abstraction would be cool.

[zachgoldstein]

Just to follow up, I went with the defaults you've laid out here in rabbit.js instead of all this patching to get exchanges. We're going to put the basic namespacing we want in our queue names. Those complications are pretty good reasons for us not to bother with the patching. So thanks for that response, much appreciated.

I don't think that sticking the reply queues on a specific exchange is really that important. Putting the request queues on a topic exchange with a routing key scheme that matches something like "service.subsystem.method" would still be nice though. I'm not 100% clear what you meant by, "Having routing in the middle breaks the REQ/REP contract". If it's just requests on another exchange, I don't think that would break it right? The point is just to have a direct exchange for the message going back from the server node. Even if it wasn't the default exchange, as long as the reply_to queue is on a direct exchange it's no biggie right?

[squaremo]

Also, setting a different exchange than the default allows us to set durability and auto-delete and all that.

The default exchange is durable, and doesn't need auto-deletion; but I take your point that you can't redeclare it with different properties.

Another thing that might be useful is being able to "tap" service requests by using a direct exchange and listening in with an additional queue.

I'm not 100% clear what you meant by, "Having routing in the middle breaks the REQ/REP contract"

I suppose the library could maintain exchanges and bindings as well as queues. My concern is trying to keep things simple. After a few extensions of the model, the various special cases and interactions between them get hard to document and maintain. Even the topic routing, added recently, is quite awkward. So I prefer to stick with simple rules and not expose much of AMQP.