tractor icon indicating copy to clipboard operation
tractor copied to clipboard

Consideration for ".start_soon()" on portals?

Open goodboy opened this issue 5 years ago • 0 comments

This is a question about our public iter-actor task spawning api.

Currently there are 2 methods which allow invoking tasks in remote actors:

Though these look similar on the surface there is differences in blocking semantics and result(s) retrieval. .run_in_actor() takes a function reference (much like the trio's .start()/.start_soon()) and returns a Portal (for which results can be obtained from .result()) while Portal.run() returns based on the semantics of the remote routine. That means when using a portal if you'd like to asynchronously call a remote async function in another actor (thereby starting a remote task) you have to use async tasks in the calling actor.

An example:

import trio
import tractor

async async_yo():
    await trio.sleep(1)
    return 'yo'

async def main():
    async with tractor.open_nursery() as n:

        # both of these return as soon as actor is initialized
        portal = await n.run_in_actor('yoyo', async_yo)
        portal2 = n.start_actor('yoyo2')

        # for 'yoyo' the task is already running at this point and we just have to wait on the result
        assert (await portal.result()) == 'yo'

        # if we invoke directly from a portal the operation is blocking until the result is received
        res = await portal2.run(__name__, async_yo)  # this blocks
        assert res == 'yo'

        # async task spawning and result retrieval using `Portal.run()`
        results = []

        async def collect_results():
            # this line is blocking which is why we've called it from a new **local** task
            results.append(await portal2.run(__name__, async_yo))

        async with trio.open_nursery() as tn:
            for _ range(3):
                tn.start_soon(collect_results)
        
        assert results == ['yo']*3

So the main question is should Portal.run_soon() exist and if so what should be the return type (if any)?

That is, should there be a way to start a remote task asynchronously using a .start_soon() style api, and further, should we also have a Portal.run() system similar to trio.Nursery.start() using a trio.TASK_STATUS_IGNORED type where the remote task can signal the "end" of the synchronous phase of the call?

Finally, I was going to ask for opinions on whether Portal.run() -> Portal.start() to indicate that the call is indeed starting a new task in a remote rpc nursery but, looking at trio modern threading api you see that from_thread.run() is the naming. I think run() is the appropriate name since it does imply not only a separate trio.run() but also that this run is taking place in another thread (or in our case a full process).

Would of course appreciate feedback from lurkers :surfer:

goodboy avatar Sep 27 '20 16:09 goodboy