how can we use the benefits of redis streams with this socket.io-redis-adapter.

[jatin510]

[darrachequesne]

I'm not sure to understand the question. Could you please explain your use case?

Reference: https://socket.io/docs/v4/redis-adapter/

[jatin510]

The redis adapater uses pub/sub feature of redis. My requirement is to replace this pub/sub mechanism of redis with the redis streams. I want to use redis streams for the chat application.

Here is an article for reference : https://get-reddie.com/blog/building-real-time-chat-with-redis-streams/

[darrachequesne]

Thanks for the details :+1:

We could indeed create an adapter based on Redis stream, instead of Redis pub/sub. If I understand correctly, the benefits would be that when a Socket.IO server looses its connection to the Redis cluster and then reconnects, it will be able to send the messages that were sent during its disconnection (they are currently lost). Is that correct?

[jatin510]

Yes. @darrachequesne that's my usecase.

I want to start integrating Redis stream with this current socketio-redis-adapter for my usecase. I am not able to figure it out. How to start this integration.

Any help will be really appreciated.

Thanks,

[darrachequesne]

We need to create another adapter, which will look a lot like this one but uses XADD instead of SUBSCRIBE.

Would you be interested to work on this?

[jatin510]

yes. I am interested in working on this new adapter. @darrachequesne

[darrachequesne]

Awesome! Could you please open a PR here, with the necessary updates?

[fractalliter]

I am working on a similar use case

[chrisandrewcl]

Another reason to use Redis streams over pub/sub is that the latter doesn't scale well (I don't have the links right now, but pub/sub performance degrades when more Redis nodes are added to the cluster, and a single Redis node can only handle so much).

Is there any work done on this front that can be shared, even if partial?

[cenzovit]

Another reason to use Redis streams over pub/sub is that the latter doesn't scale well (I don't have the links right now, but pub/sub performance degrades when more Redis nodes are added to the cluster, and a single Redis node can only handle so much).

Is there any work done on this front that can be shared, even if partial?

I have been looking into the scaling of Redis pub/sub as I attempt to scale a node application that relies on socket.io, and it would appear to me that the horizontal scaling of pub/sub has been addressed in Redis 7.0 (see: https://github.com/redis/redis/issues/2672 and https://github.com/redis/redis/pull/8621 and https://redis.io/docs/manual/pubsub/#sharded-pubsub). Is there any work being done in the socket.io-redis-adatper to support the use of pub/sub sharding via SSUBSCRIBE, SUNSUBSCRIBE and SPUBLISH?

[cenzovit]

@jatin510 @darrachequesne, any update on the creation of a socket.io-redis-streams-adapter or any thoughts on supporting pub/sub sharding within the socket.io-redis-adapter? It seems to me that until one of those things is implemented/released, the current implementation is incapable of scaling horizontally (it actually appears to scale negatively as the redis cluster increases in size).

[darrachequesne]

@cenzovit that's in my backlog, I'll keep you updated.

[darrachequesne]

Here we go: https://github.com/socketio/socket.io-redis-streams-adapter

Feedback is welcome!

[darrachequesne]

For future readers:

Please see the documentation here: https://socket.io/docs/v4/redis-streams-adapter/