Allow for custom hook callers?

[goodboy]

In #133 there was a request to be able to provide a custom _HookCaller (in this case in order to be able to warn on deprecated hook calls - since addressed via #138).

This feature may be quite useful in the future not only in its ability to allow users to customize the pluggy runtime but also if we want to eventually support async / coroutine based plugins (this can be done loop agnostic these days with something like multio and/or trio-asyncio).

I'd like to get feedback on this.

To start it would mean that we would make a simple change where PluginManager can take a hook_caller_type as input during instantiation.

[RonnyPfannschmidt]

i propose deferring this until we can experiment against trio and asyncio

based on initial example i'd absolutely avoid multio

[RonnyPfannschmidt]

oh - we might want to investigate removing multi-call by moving it into a hook-caller subclass/mixin

[nicoddemus]

How about multiple hook callers? At work we might need to use a pluggy hook call system identical to pluggy, but also capable of loading hook implementations from shared c++ libraries.

cc @williamjamir

[goodboy]

Love both of those ideas!

[RonnyPfannschmidt]

@nicoddemus sounds like overkill a hook parser or a wrapper sounds more viable there

[njsmith]

It's already possible to define a synchronous hook that internally does trio.run(something_async) or asyncio.run(something_async). To get extra value from async hooks, I think you'd need to somehow "expose" the async-ness to the caller, so that the caller can be async, and do things like:

async def run_test():
    await run_hooks("setup-fixtures")
    await run_hooks("run-test")
    await run_hooks("teardown-fixtures")

trio.run(run_test)

so that multiple hooks are running under the same event loop.

In principle I'm guessing this wouldn't be too complicated. But there's not much point unless one of pluggy's users (like pytest) is interested in using it in this mode.

[justanr]

@njsmith I've toyed with this idea and what I ended coming up with is that hookimpls essentially become coroutine factories and the caller awaits their results, that way you can call from it inside another coroutine without pissing off the event loop (not sure how trio or curio handle the equivalent of loop.run_until_complete from inside a running coroutine, but I imagine they don't take kindly to it).

I never expanded it beyond toy examples so I'm not sure what shortcomings I missed beyond the caller needs to reimplement some of pluggy's filtering logic.

[RonnyPfannschmidt]

i had a sync with @simonw on the weekend and i will try to incooperate the details into pluggy as i sprint on speedups this week