Feature Request: Raft backend for retained messages

[aclerici38]

Hi there, I am running rmqtt clustered via rmqtt-cluster-raft and would like to enable retained messages. Currently via the rmqtt-retainer plugin redis is required for this. This seems redundant and introduces an unnecessary external dependency as each replica is already in a stable sync via Raft, would it be possible to replicate the retained messages via Raft as well?

e.g. if storage.type is sled or ram in a Raft cluster, rmqtt is able to use that backend for retained messages but the retained messages are also replicated across the cluster via Raft.

Thanks!

[bittcrafter]

Thanks for the suggestion — this is indeed a constructive idea. We’ve also thought about whether retained messages could be replicated directly via Raft. However, there are some concerns:

1. Plugin independence: The cluster-raft plugin and the retainer plugin are designed to be independent modules. Directly coupling retained message synchronization with Raft in the cluster plugin would break this separation and make maintenance/extension more complex.
2. Replication overhead: In a clustered setup, publish messages are already synchronized across nodes. Adding retained message synchronization via Raft would further increase the replication load, which could negatively affect overall cluster health and performance.
3. Snapshot size and stability: Storing retained messages in Raft state would enlarge snapshots, making them heavier to create, transfer, and restore. This may impact stability and recovery speed in larger deployments.

That said, your suggestion makes sense from the perspective of reducing external dependencies. We’ll keep this in mind and evaluate if there’s a way to provide a more lightweight option for retained messages in clustered deployments, without introducing the above drawbacks.

[aclerici38]

Thanks for the detailed response. I'll take your word for points 2 and 3 as I do not have much experience with large mqtt or raft deployments. My deployment is very small and these would not affect me, but I understand there are lots of use-cases to consider here.

For point 1, are the cluster-raft and retainer plugins currently independent? It seems to me the storage type in the retainer plugin is directly dependent on whether the cluster-raft plugin is enabled, so while they operate independently the behavior does seem dependent. In my opinion, allowing raft replication for retained messages will make the 2 plugins independent as all storage types for the retainer would be allowed whether or not the cluster-raft plugin is enabled, just with a new code-path to replicate via raft as well.

[onedr0p]

From my perspective it doesn't seem like you're going to use raft on a single node deployment, so it would be neat if raft could also handle retention and replication.