Native MQ transport (aka support for scalability protocols)

[goodboy]

As has always been the plan (because it fits the actor model so well) we need native zeromq support. This will likely need a little thinking and re-design to integrate with the current Channel api.

Some prerequisites before this can move forward:

• native trio support; @njsmith has some notes in trzmq in order to make this work alongside pyzmq.
• an alternative approach might be writing a more from scratch lib more like aiozmq for which there is relevant discussion here.

[njsmith]

So there's two ways to integration zmq with asyncio:

1. Define a custom asyncio event loop that has special support for zmq, and then use this everywhere.
2. Use zmq's annoying and error-prone notification API to get it working on top of a normal asyncio event loop.

pyzmq implements all the low-level bindings you need to do anything with zmq in Python, and then also provides an implementation of option (1).

aiozmq uses pyzmq's low-level bindings, but then implements option (2) on top of them.

In Trio, option (1) is not possible. So we have to do the equivalent of option (2), where we use pyzmq's low-level bindings and hook them up to Trio's built-in event loop. That's what the notes in the trzmq repository are about. (This is also why trzmq is not a totally trivial thing. I don't think it should be too hard, I believe the notes there do describe all the issues, but it took a while to figure that out, which is why I haven't actually finished it :-).)

[goodboy]

@njsmith thanks for the input :+1: I'll be trying to take a look at trzmq again hopefully soon!

[goodboy]

After looking at the trials involved with getting trzmq working I've instead opted to experimenting with nanomg ng (aka nng) for which there is a new python wrapper pynng being developed with (hopefully) native trio support in codypiersall/pynng#4.

The details of why nng was built to replace nanomsg are interesting and also why nanomsg was built to improve upon ZeroMQ.

[goodboy]

New work on trzmq also going on: https://github.com/remleduff/trzmq/commit/ce72ac5f5e3ae34dc4df349abe6f33e113ee2be2

Simpler then expected. Gonna wait to see what core trio devs say about it.