server
server copied to clipboard
mochi-mqtt
Cluster MQTT broker based on mochi-co/mqtt
I was looking for cluster ready mqtt broker for my home environment (k8s) for some time already. But unfortunately all i checked (emqx, vernemq) have some issues (e.g not compatible with Paho, or broken in home environment).
So based on mochi-co/mqtt i build my own broker - i was heavily inspired by https://github.com/mochi-co/mqtt/pull/45
Maybe other people looking for clustering capabilities will be interested, or even better some pieces can be used inside mochi-co/mqtt!
If you think that this is not a place for this kind of communication, please close :)
https://github.com/bkupidura/broker-ha
Amazing! This is very interesting, and I like what you've done here... I will definitely look into it deeper when I get some time and have a think about how we might implement some of these features in the future :)
Cluster is implemented using gossip and raft protocol. https://github.com/wind-c/comqtt based on mochi-co/mqtt .
@anthonycorbacho probably you are more interested in clustering capabilities directly in mochi-co/mqtt, but i just pushed new version to bkupidura/broker-ha which now is based on mochi-co/mqtt v2.
@bkupidura I've been wondering recently if it would be possible to build clustering functionality using the hooks system, but I haven't looked too much into cluster myself. I wonder if it is possible?
@mochi-co short answer would be yes, it should be possible ;) Currently broker-ha is using mochi-co/mqtt hooks to forward PUB messages to different members. But my implementation depends on hashicorp/memberlist.
Memberlist is used to exchange data between cluster members. It can be replaced by any protocol. Initially i was trying to build this on top of vanilla MQTT. But this approach can be little tricky and complicated with 2+ cluster members. My initial idea was similar to mosquitto bridge configuration. But this requires mqtt client connected to every broker and syncing state between them over MQTT... Memberlist approach looks much simpler.
You would need to solve 3 main issues:
- How to discover cluster members? (DNS SRV discovery in broker-ha)
- How to handle cluster member join/leave? (memberlist in broker-ha)
- How to exchange data between cluster members? (memberlist in broker-ha)
Thanks @bkupidura, I will have a think about this and see if I can come up with any ideas and let you know. It would be very interesting if we could make a hook based cluster feature.
I implemented cluster based on mochi-co/mqtt v2.0, and mochi-co's hook design is easy to expand. However, a few changes were made to the mochi-co/mqtt code. Cluster is complex and involves node discovery, data consistency, stability, and performance.I use gossip for node discovery and raft for data(subscribe/unsubscribe) consistency synchronization, and use redis to store inflight, retain messages and subscriptions across nodes.Publish messages are routed to the corresponding node according to the subscription table. Interested developers can refer to.
This is more of a clarification question. Is the desire to have a masterless cluster (node.1 <-> node.2 <-> node.n) or something like a more traditional cloud model (LB -> scaling group)
This discussion is now located at https://github.com/orgs/mochi-mqtt/discussions/280#discussioncomment-6755488