pluggy icon indicating copy to clipboard operation
pluggy copied to clipboard

planning towards supporting fully mixed sync/async hook calling

Open RonnyPfannschmidt opened this issue 2 years ago • 6 comments

cc @simonw

this is a rough outline of supporting async hook calls and awaitable results in

the starting point will be a async hook caller that will call sync and async hook, deferring all awaitables to the event loop

nesting like async->sync->async will trigger failures when awaitables are encountered in the inner loop

the next level will be deferring all sync calls to a thread executor any sync call that triggers a awaitable, will steal one thread of that pool , this is considered acceptable as execution is typically expected to be fairly non-concurrent (based on pytest)

a own threadpool with a maxium size of 10 ? shal be used (this accounts for the expectation that hooks run in order and should not nest to arbitrary levels)

nonblocking sync hooks may be marked as nonblocking in which case they may run in the mainloop, as before nesting async into sync will trigger errros

the primary use case for nonblocking sync hooks is to support being nested into other legacy hooks without causing issues but transparently working with modern hook calls

RonnyPfannschmidt avatar Mar 23 '23 10:03 RonnyPfannschmidt

Why is mixing sync and async hooks needed?

bluetech avatar Jun 17 '23 20:06 bluetech

To migrate ecosystems

RonnyPfannschmidt avatar Jun 17 '23 22:06 RonnyPfannschmidt

Is this intended to support only asyncio, or open to other async libraries?

nicoddemus avatar Jun 18 '23 12:06 nicoddemus

Any async, end goal here is to prepare the possible recoloring of the pytest internals

RonnyPfannschmidt avatar Jun 18 '23 13:06 RonnyPfannschmidt

Hello, First thanks for the great framework. Just wondering if this subject is still planned to be worked on ? Or is this “await me maybe” pattern approach good enough ?

toutantic avatar May 14 '25 07:05 toutantic

Its still planned

Its just requiring way more work than i hoped

Recent restructuring makes it a bit more easy

However mixed coloring is very hard and requires fundamental changes to all parts of the system

Partially due to the need to enforce correctness invariants

Else people using the api Wrong by accident will miss grave error conditions

RonnyPfannschmidt avatar May 14 '25 07:05 RonnyPfannschmidt