alibaba
Photon and ZeroMQ
Hello there:
ZeroMQ sockets could be used to facilitate implementing variety of scenarios easily. So for testing purposes, the integration with Photon has been done successfully at first attempt. But for more complex cases, a cascading event io has been used to integrate multiple ZMQ router sockets. The relevant file descriptors also have been registered accordingly. The master engine is uring, cascading is epoll and photon event loop used for notifications. It would be helpful to know that a fd_context has been developed which is a base of NotificationContext to make notifications faster without checking each event data as are out of order. A tiny client has been developed for testing purposes as a dealer which is a good fit for router socket. But the event engine is behaving somehow strange. The read event has been notified successfully but when a single read operation used to receive frames from router socket in none blocking mode, first frame is route id and second frame is actual data, then nothing is received. The event loop also checked for errors and returns used for event loop operation. If read operation has been done twice sequentially then it works. The integration nicely works without cascading.
Any feedback would be welcome.
Thank you for all these. I likes ZeroMQ and used to use it a lot, your work is definitely welcomed to Photon.
About the event notification problem you have met, seems like something goes wrong. You might want to try it with epoll master engine with epoll cascading engine to see if it will happen. And, since our event engine only watches fd status, once data kept by zmq buffer, photon will not able to know, you might need to check if zmq read event is as same as file descriptor event, or data received from socket have stored in buffer and related event is just zmq status but not fd status.
Maybe you can put your code somewhere so we may able to try finding out what causes such lack-of-event scene.
Maybe you can put your code somewhere so we may able to try finding out what causes such lack-of-event scene.
+1 We can find out if there's some mis-understanding about the semantics of cascading engine's interface.
The integration nicely works without cascading.
So, why do you use cascading?
FYI, I took a look in photon structure design comparing to operating systems concepts, design, threading models and synchronisation mechanisms. Our project has been developed and tested by ASIO at first attempt but the results are not satisfactory based on requirements. So switched to photon as a replacement according to benchmarks and the features we're looking for considering its size, performance, scalability, utilisation, features and taking operating system concepts in mind.
Integrating ZeroMQ and Photon requires specific considerations such as ZeroMQ event engine which is edge-triggered, non standard sockets with underlying pipes and real events dispatching. I have tried epoll cascading engine also but that is not the source of problem. The reason behind using cascading engine is the broker design pattern consisting of different router sockets. The broker is one of the important components of our project which requires particular attention. And it is not only about broker but distributed services. Ok.
For integration purpose, I did not want to make any modification to both libraries but use provided features of both worlds to do an elegant fusion. So, I took photon ETKernelServer as a template and developed a simple ZeroMQ socket wrapper named "fd_context" as a base of "NotificationContext" to be registered in poller which is used in photon event loop for events notification of edge-triggered mode. But the problem is about ZeroMQ real events dispatching. Epoll cascading engine just discloses number of events happened on ZeroMQ sockets and not the real events. Even epoll behaves similarly. And in next iterations no events will be reported because of edge triggered mode but there are. Anyhow, this is not a problem if ZeroMQ non blocking operations is going to be used in DOIO_ONCE with waiting for fd readable by using native socket descriptors of ZeroMQ. But there would be issues which required to paid attention to. Nevertheless, it would be possible to use ZeroMQ poller or zpoller also. But I would prefer integrating with photon event loop.
The source code is not still ready to publish. It is under development and will be provided for any sort of comments and recommendations asap. We are under attack by Israel and USA but the life is going on.
Wish you health and happiness.
We are under attack by Israel and USA but the life is going on.
😲 hope it's going to end soon.
non standard sockets with underlying pipes and real events dispatching
What do you mean by non standard sockets with underlying pipes and real events dispatching?
BTW, regarding MQ, it is said that Rocket is awesome.
sockets with underlying pipes
Are you spliceing buffers to achieve zero-copy?
We are under attack by Israel and USA but the life is going on.
😲 hope it's going to end soon.
non standard sockets with underlying pipes and real events dispatching
What do you mean by
non standard sockets with underlying pipesandreal events dispatching?BTW, regarding MQ, it is said that Rocket is awesome.
ZeroMQ is widely used because of its sockets used to solve variety of problems in networking and lock free communication. By non standard it means it is not possible to get access to a socket descriptor of a peer which connected to a router socket, for example. It does not provide access to underlying socket descriptors. But it is possible to get socket descriptor of router only and register it in epoll for event notifications which is not reliable to use its descriptor directly in an io operation. Events Notification is not reliable because of edge-triggered mode. So, by real events, it means we need to use ZeroMQ get events interface to get the events encountered on a router socket as we cannot rely on events reported by epoll cascading engine for example. If you take a look at AZMQ which is a ZeroMQ port for ASIO, you can find out more about that.
sockets with underlying pipes
Are you
spliceing buffers to achieve zero-copy?
ZeroMQ supports zero copy also. It is achievable even in multipart messages. Instead of copying a buffer to ZeroMQ message, it is possible to transfer buffer ownership to prevent buffer copy.