nng
nng copied to clipboard
nanomsg
Publisher side filtering
Is your feature request related to a problem? Please describe.
My case like the follow picture:
problem:
- Even if I set the topic, each subscriber will receive all data publisher pushed in the net.
- If the data flow is sufficient, the subscriber cannot increase the processing capacity through horizontal scaling.
Describe the solution you'd like Is it possible to provide the ability to set a topic against the subscribe on the Publisher side?
Describe alternatives you've considered
Additional context
What you want is publisher side filtering. This is not possible with the current protocol. I do have plans for a new protocol that would be able to do this but I cannot estimate when it will arrive in the code as I'm working on a bunch of other priorities first - and NNG is not a full time effort for me.
Sounds like a great improvement! Is there another open issue about publisher-side filtering that I could track?
My suggestion would be not to add this as an extension to the protocol, but just to have the publisher application add an additional pub/sub that only republishes filtered message using any filter / logic wished for. The remote end can than subscribe to the filtered publisher.
With aio it is very lightweight to add the filtered publisher and you get all the flexibility for filtering including a dedicated queue for the filtered topic.
For each specific filter you would get a dedicated second publisher + queue.
So my view is that this is best solved in the application and not in NNG it self.
That approach is a reasonable workaround, but as it requires creating new endpoints, it isn't very conducive to easy administration at scale. I consider the request still outstanding for now.
MQTT does this of course via a broker. But it is useful even in brokerless configurations.
Brokerless doesn't give you reliability - but it would support better scalability and could be extended into a hypothetical multicast transport.