Mechanism to subscribe to additional channels on WebSocket connection

[jchv]

For at least WebSocket connections, it is possibly desirable to allow a connection to subscribe to additional channels without closing and reconnecting to the WebSocket. In particular, I have a use case where this would be important. Does it make sense for nchan to support this use case?

[slact]

Does it make sense for nchan to support this use case?

I think it does. However, I have yet to find a good interface to do with, without introducing subscriber ids and control/admin locations. If you have any suggestions, I'll be glad to hear them.

I would also like to know the details of your use case.

[jchv]

I have no suggestions unfortunately :( I've given it some thought, but it indeed seems difficult. It seems like some things may need to be redesigned at least to some degree for this to work, and I do not have intimate enough knowledge of nchan to provide any good insight on that yet.

I actually have multiple use cases where I am considering nchan, but the one I'm thinking of in particular, to be vague, involves short but very frequent messages (think under a kilobyte or so) that occur over tens of thousands of channels, that a user may subscribe to many at once and may rapidly subscribe and unsubscribe. I am not sure if nchan is a perfect fit, but it seems to be quite efficient enough to handle this kind of use case.

[slact]

I do have a mechanism in mind for switching subscriptions with a no-loss guarantee, but it still involves reconnecting. Basically, A subscriber could disconnect, then connect with the previous subscription's URL and message-id passed in the headers. Something like:

GET /subscribe/foo,bar,baz
X-Nchan-Previous-URL: /subscribe/foo,bar
X-Nchan-Previous-Message-ID: 194832343:-,[0]

This would ensure that the subscriber reconnects to channels it used to subscribe to without losing or repeating any messages.

This approach still requires a reconnect, so this is not optimal for a heavier protocol like Websocket, and perfect for something like EventSource.

---

As for your use case, tens of thousands of channels is no problem at all. Actually, the more channels, the more efficiently Nchan works, because load is distributed between Nginx workers on a per-channel basis. The only limit here is one subscriber's connection can subscribe to at most 255 channels.

[michaelfig]

Would it be possible to implement this reconnect-with-no-loss feature entirely in terms of the proxy_pass service? Does it just have to munge the last_message_id before sending the X-Accel-Redirect, and if so, how?

Thanks, Michael.

[mojtabye]

same issue, i also asked here https://stackoverflow.com/questions/51806034