High Latency in Pub/Sub Mode with Increasing Subscribers (Single Publisher, Hundreds of Subscribers)

[Zunea]

May I ask why, in the pub/sub model, the latency increases as the number of subscribers increases? How can this issue be resolved? I’m trying to implement a single-writer, multi-reader setup (one publisher, hundreds of subscribers). Thank you very much!

[elfenpiff]

@Zunea, can you give us a bit more details about your overall use case? Why do you need hundreds of subscribers? This seems to be a very exotic use case - maybe a different messaging pattern could be the solution here.

Nevertheless, I can confirm that the latency increases with the number of subscribers. Those are the measurements from my laptop:

• 10 subscribers - ~400ns
• 100 subscribers - ~3000ns
• 200 subscribers - ~5700ns

I think we cannot prevent the latency from increasing with the number of subscribers, but we should be able to optimize the code further. Until now, we have not yet put any effort into performance optimizations - the gains we got from classic iceoryx to iceoryx2 were caused by an improved architecture.

[Zunea]

@elfenpiff

Thank you for your response!

My use case is high-frequency trading. We have a market data process that centrally subscribes to WebSocket feeds and needs to forward the data to multiple subprocesses. I want to use the pub/sub pattern to replace busy-loop polling in order to improve server resource efficiency. From my perspective, iceoryx2 seems to be the most suitable solution for this architecture.

I also have a question: Is the active notification mechanism in iceoryx2 implemented based on POSIX semaphores?

[elfenpiff]

@Zunea

We have a market data process that centrally subscribes to WebSocket feeds and needs to forward the data to multiple subprocesses.

For this, pub/sub seems like the correct messaging pattern, but maybe you could create one service per feed or ticker symbol. This would massively reduce the number of subscribers. (I assume that your payload looks a bit like a struct consisting of a float and a field for the ticker symbol). iceoryx2 does not have a problem with creating a massive amount of services, and Linux should also be fine with it (We tested this some time ago for 100.000 services on one laptop, and it worked fine - without any reduced latency).

Is the active notification mechanism in iceoryx2 implemented based on POSIX semaphores?

Actually, it is unix datagram sockets (for IPC) and socketpairs (for local) events. The lower building blocks also provide semaphore-based triggering, which is, in fact, ~10% faster, but semaphores cannot be attached to an event multiplexer like select or epoll; therefore, we did not use them on the end-user API. We have a plan to integrate them there, but then you are no longer able to use the WaitSet when you go for a semaphore-based event mechanism.