tractor icon indicating copy to clipboard operation
tractor copied to clipboard

Interloop (`trio` <-> `asyncio`) memory channel design

Open goodboy opened this issue 4 years ago • 0 comments

With the completion of #121 we now have a way to wrangle asyncio code from SC-actor-land.

One of the obvious ugly parts of the current api is the delivery of an asyncio.Queue to the target asyncio task. In keeping with trio style it would be much better to offer a small abstraction much like trio.abc.Channel to make asyncio-side streaming semantics more elegant.

We already introduced a LinkedTaskChannel as part of #121 for the trio side and it would be handy to have something equivalent for asyncio tasks.


Some design options:

  • make two types of LinkedTaskChannels, one for each task in each loop
    • probably allows the most flexibility for the apis allowed from either task and is most explicit type-wise?
  • universal queue ideas from the trio peeps:
    • orig by njs: https://gist.github.com/njsmith/3494ebd27f32c2ac168c71de116ccc5d
    • fixes from richardsheridan: https://gist.github.com/richardsheridan/8f803a5a15831840846083a6cbcefbea
  • adopt either directly or by inspiration some stream apis from anyio?
    • we could adopt a stapeled stream style where we dynamically figure out which loop the channel object is being used in and then runtime-define the methods implementations? This might allow for a more symmetric channel api/object at the expense of some magical method definitions?
  • consider adding a .started() method to whatever the asyncio-side channel type ends up being much like the anyio.from_thread.BlockingPortal.start_task()/trio.Nursery.start() target function mechanisms. This would allow us to avoid the current manual first-sent-value-ublocks-the-trio side to_asyncio.open_channel_from() entry.

Very open to and interested in other ideas (and further discussion) from the lurkerz.

goodboy avatar Dec 12 '21 00:12 goodboy