pebble icon indicating copy to clipboard operation
pebble copied to clipboard

Context in done_callbacks

Open gg-mmill opened this issue 2 months ago • 3 comments

Hello, I noticed that contextvars are not passed inside the task_done callback of a future from a ProcessPool.

I haven't investigated this much yet, but do you know if this is expected / wanted ?

As a workaround, I can copy the context and pass it to the callback, then use Context.run to take it into account, but it's a bit cumbersome.

Happy to have your thought on this, and provide a reproducible test case if needed

gg-mmill avatar Oct 30 '25 15:10 gg-mmill

Hello,

could you please provide code examples on how you are using Context Variables in your application complete with a description of what you would expect to see and what happens instead?

noxdafox avatar Nov 01 '25 22:11 noxdafox

Hello, the Context variables are mostly used by structlog (see https://www.structlog.org/en/stable/contextvars.html#context-variables)

For example this snippet

from pebble import ProcessPool
import structlog

logger = structlog.get_logger("worker")

def worker_fun(arg):
    logger.info("task done", arg=arg)

def callback_fun(future):
    logger.info("callback")

with structlog.contextvars.bound_contextvars(context="some context"):
    with ProcessPool() as pool:
        for i in range(2):
            future = pool.schedule(
                worker_fun,
                args=[i],
            )
            future.add_done_callback(
                callback_fun
            )

generates the following logs:

2025-11-04 16:49:43 [info     ] task done                      arg=0 context='some context'
2025-11-04 16:49:43 [info     ] task done                      arg=1 context='some context'
2025-11-04 16:49:43 [info     ] callback                      
2025-11-04 16:49:43 [info     ] callback                      

But I would have expected context='some context' to also be present in the logs in the callbacks.

gg-mmill avatar Nov 04 '25 15:11 gg-mmill

Callbacks are executed in a separate thread from the main one. Hence, you will see a different context during their execution.

noxdafox avatar Nov 15 '25 23:11 noxdafox