Weak tasks (auto-cancel when no normal tasks are running)

[Tronic]

There are cases where a "monitoring" or "connection pool" task needs to be run within an application's nursery as long as there are other tasks running but it would be more convenient to have them automatically cancelled as soon as their work is no longer required. Such a task can be implemented by making it monitor its parent nursery and self-cancel once no other tasks are running. However, such solution fails if any other task in the same nursery uses similar approach, as they'll see each-other and will never terminate.

I am wondering whether there would be enough use for such feature to justify adding it to Trio core, where it can be reliably detected that only weak tasks are running, also avoiding the need for separate watchdog tasks inside each of the said tasks.

async with trio.open_nursery() as nursery:
    db = Database()
    await nursery.run(db.connection, weak=True)
    nursery.run_soon(important_job, db)
    nursery.run_soon(another_important_job)
    nursery.run_soon(progress_monitor, weak=True)

Whenever a task terminates, Trio would check if only weak tasks are left, and in that case issue nursery.cancel_scope.cancel(), terminating db.connection and progress_monitor (if they are still running).

The same feature could be used for run_race style constructs by marking all tasks weak so that they get auto-cancelled as soon as the nursery body receives the first result and exits.

Maybe there is something fundamentally flawed with this idea, or perhaps there already is a convenient way to handle such cases, without building manual cleanup logic.

[njsmith]

The quick hack for when this comes up is:

async with open_weak_nursery() as weak_nursery:
    async with trio.open_nursery() as strong_nursery:
        db = Database()
        await weak_nursery.start(db.connection)
        strong_nursery.start_soon(important_job, db)
        strong_nursery.start_soon(another_important_job)
        weak_nursery.start_soon(progress_monitor)
    # After strong_nursery block exits:
    weak_nursery.cancel_scope.cancel()

You could wrap the pattern up into a utility function, like:

@asynccontextmanager
async def open_weak_nursery():
    async with trio.open_nursery() as nursery:
        yield nursery
        nursery.cancel_scope.cancel()

Or even:

@asynccontextmanager
async def open_weak_and_strong_nurseries():
    async with trio.open_nursery() as weak_nursery:
        async with trio.open_nursery() as strong_nursery:
            yield weak_nursery, strong_nursery
        weak_nursery.cancel_scope.cancel()

# Usage:
async with open_weak_and_strong_nurseries as (weak_nursery, strong_nursery):
    ...

That still leaves open the question of whether we should provide some utilities like this with trio by default. But I think it's persuasive that we don't need to add anything into the core nursery implementation itself, and that adding this isn't super urgent.

[wgwz]

I just wanted to add a link to #472 here because it as well as #569 are definitely related to this question. I found this pattern really helpful myself for something I was working on in hopes of contributing some worked examples for in 472. As already mentioned this pattern should definitely be highlighted in the docs.