asyncstdlib icon indicating copy to clipboard operation
asyncstdlib copied to clipboard

Helper to emulate `async yield from`?

Open maxfischer2781 opened this issue 4 years ago • 1 comments

Asynchronous generators have no (async) yield from to delegate to sub-iterators (see yield from PEP 380). asyncstdlib itself frequently has to use loops to work around this, but this is cumbersome and slightly incorrect (see the PEP for details).

A trampoline could be used to emulate async yield from by switching between async generators:

@yield_fromable  # wrapper to switch between main and nested generators
async def async_gen2():
    await asyncio.sleep(0)
    yield async_from(async_gen())  # schedule nested generator
    await asyncio.sleep(10)  # busy farming SO rep

maxfischer2781 avatar Apr 15 '21 12:04 maxfischer2781

The inner marker can in principle come in two flavors, sync (yield from) and async (async yield from). A single async neutral marker might suffice.

It appears there are four feature stages for the trampoline:

  1. Iterable forwarding: just call yield await nested.__anext__() until completion
    • Equivalent to the async for item ...: yield item loop
  2. Value forwarding: repeatedly call v = yield await nested.asend(v)
  3. Return forwarding: if the nested iterator returns (via StopIteration) pass the result to the outer generator
  4. Exception forwarding: if yield throws do nested.athrow. if nested.athrow/nested.send throws do outer.throw.

It might be worthwhile to implement these stages separately, to switch off complicated code when it is not needed. Might also allow a faster first release.
Point 1. is the simplest, and probably what most cases need. 2. and 3. should be roughly the same complexity. 4. seems very complex, requiring to switch between throw/send/__anext__ paths.


Regular coroutine signal forwarding should work in any case, since await does the forwarding. This should be tested, though.

maxfischer2781 avatar Apr 15 '21 12:04 maxfischer2781

Closing this as out of scope. A custom implementation would likely be a) huge and b) slow, so little benefit at quite a cost.

maxfischer2781 avatar Mar 20 '23 15:03 maxfischer2781