channels_redis icon indicating copy to clipboard operation
channels_redis copied to clipboard

PubSub layer violates Channels Layer Spec

Open jalaziz opened this issue 3 years ago • 0 comments

While attempting to fix #270, I realized that the PubSub layer doesn't actually conform to the Channels Layer Spec.

In particular, the spec says:

Channels are a first-in, first out queue with at-most-once delivery semantics. They can have multiple writers and multiple readers; only a single reader should get each written message. Implementations must never deliver a message more than once or to more than one reader, and must drop messages if this is necessary to achieve this restriction.

While the PubSub layer is "at-most-once", it doesn't guarantee that a message will only be delivered to a single reader. This has implications for workers as having multiple workers listening to the same channel will result in duplicate messages.

A potential solution is to introduce a Redis Streams layer which allows such guarantees. The challenge with Redis Streams is cleaning up messages to ensure "at-most-once" and to keep the memory usage sane. On the bright side, it would re-introduce some of the fault tolerance that was lost with the PubSub layer.

jalaziz avatar Mar 06 '22 10:03 jalaziz