azure-functions-python-worker icon indicating copy to clipboard operation
azure-functions-python-worker copied to clipboard

[Bug] dispatcher.py task factory doesn't allow for optional `context` parameter

Open jreed-Aces opened this issue 1 year ago • 5 comments

Expected Behavior

Python 3.11 allows for a named context parameter to be passed in when constructing a task (https://github.com/python/cpython/blob/3.11/Lib/asyncio/tasks.py#L106). However, the task factory that creates a ContextEnabledTask inside of dispatcher.py doesn't allow for that parameter. When a task is added that attempts to utilize the parameter, an error is raised instead:

System.Private.CoreLib: Exception while executing function: Functions.mention_queue_trigger. System.Private.CoreLib: Result: Failure
Exception: TypeError: Dispatcher.dispatch_forever.<locals>.<lambda>() got an unexpected keyword argument 'context'
...
File "~\Python\Python311\Lib\asyncio\base_events.py", line 449, in create_task
    task = self._task_factory(self, coro, context=context)

Actual Behavior

The task_factory lambda and the ContextEnabledTask should be able to accept the optional, named parameter of context

Steps to Reproduce

Using the existing task_factory, attempt to add a new task along with the named context parameter.

Relevant code being tried

The following fix appears to resolve the issue:


+++ b/azure_functions_worker/dispatcher.py
@@ -141,7 +141,7 @@ class Dispatcher(metaclass=DispatcherMeta):
                         worker_id=self.worker_id)))

             self._loop.set_task_factory(
-                lambda loop, coro: ContextEnabledTask(coro, loop=loop))
+                lambda loop, coro, context=None: ContextEnabledTask(coro, loop=loop, context=context))

             # Detach console logging before enabling GRPC channel logging
             logger.info('Detaching console logging.')
@@ -849,8 +849,8 @@ class AsyncLoggingHandler(logging.Handler):
 class ContextEnabledTask(asyncio.Task):
     AZURE_INVOCATION_ID = '__azure_function_invocation_id__'

-    def __init__(self, coro, loop):
-        super().__init__(coro, loop=loop)
+    def __init__(self, coro, loop, context=None):
+        super().__init__(coro, loop=loop, context = context)

         current_task = asyncio.current_task(loop)
         if current_task is not None:



### Relevant log output

_No response_

### requirements.txt file

_No response_

### Where are you facing this problem?

Local - Core Tools

### Function app name

_No response_

### Additional Information

_No response_

jreed-Aces avatar Jun 10 '24 17:06 jreed-Aces

@jreed-Aces have you found a workaround for this? I'm having the same issue right now and you just happened to create this 2 days ago 🙂

brbarnett avatar Jun 12 '24 18:06 brbarnett

@brbarnett We downgraded to Python 3.10, which doesn't have the optional context parameter. It fixes the issue, but it's far from ideal.

Locally the code fix I posted above works, but that doesn't help when deploying to Azure.

jreed-Aces avatar Jun 12 '24 19:06 jreed-Aces

I have the same issue with py 3.11... Fixed for the moment with py 3.10 version

Darcos923 avatar Jun 13 '24 19:06 Darcos923

ran into the same issue with python 3.11 when working on http stream mode on azure function app

dslam avatar Jun 21 '24 21:06 dslam

run into the same issue using langchain Runnable method ainvoke. This worked for me but don't know why:

loop = asyncio.get_running_loop() loop.set_task_factory(None)

TobiConti avatar Jun 24 '24 12:06 TobiConti

Any update on this? I see there's an open PR to solve it since more than a month ago? And @TobiConti's solution doesn't really work for me within an Azure functions environment

MarthinusBosman avatar Aug 08 '24 17:08 MarthinusBosman

The fix has been merged and will be in the next worker release. Thanks for your patience!

hallvictoria avatar Aug 14 '24 19:08 hallvictoria

The fix has been merged and will be in the next worker release. Thanks for your patience!

any timeline for the next release?

tmusbi avatar Aug 19 '24 19:08 tmusbi

Hi @tmusbi, sorry for the late response! We've hit some delays with getting a new release out, but we hope to have this out by late October / early November. Thanks for your patience!

hallvictoria avatar Sep 30 '24 20:09 hallvictoria

Posting a quick update here -- unfortunately we've hit some other delays in releasing, and we are slowing releases during the holiday season. Updated ETA is January. Sorry for the delays, and thanks again for your patience.

hallvictoria avatar Nov 15 '24 23:11 hallvictoria

any timeline for the next release?

santiagovasquez1 avatar Dec 26 '24 18:12 santiagovasquez1

any timeline for the next release?

Try the comment right before yours.

MarthinusBosman avatar Dec 26 '24 18:12 MarthinusBosman

I see that the ticket is closed, but I don't see any comments about release. Has this been released, or is it still waiting?

jreed-Aces avatar Jan 17 '25 13:01 jreed-Aces

@jreed-Aces, the release will start next week. The changes should be rolled out to all regions by end of February / early March and are included in runtime version 4.1037.

hallvictoria avatar Jan 17 '25 15:01 hallvictoria

@hallvictoria great, thank you for the update. I appreciate it

jreed-Aces avatar Jan 17 '25 16:01 jreed-Aces

run into the same issue using langchain Runnable method ainvoke. This worked for me but don't know why:

loop = asyncio.get_running_loop() loop.set_task_factory(None)

This also helped me

TJKlein avatar Feb 09 '25 18:02 TJKlein

Hi @hallvictoria, thank you for your work and availability, I see that Function Apps runtime version 4.1037.0 has been released but I couldn't find an explicit mention for this bug there. Pardon me if I missed it.

Can you please confirm it's included in the release, and migrating back to this runtime is safe?

In the meantime, +1 to @jreed-Aces 's suggested workaround. I deployed a new container with Python 3.10 2 months ago and it works like a charm. I'm just preoccupied about tech debt on this old runtime.

Thanks for your time and efforts!

robincolinet avatar Mar 11 '25 14:03 robincolinet

Hi @robincolinet, yes, this fix is included in the 4.1037 train. 4.1037 is still being rolled out but will be available for all regions by the end of next week at the latest.

The runtime release notes will show updates to the Python worker version. You can view the changes for the Python worker in the releases page, and it will list what runtime version the changes will be a part of. Hope that helps!

hallvictoria avatar Mar 11 '25 15:03 hallvictoria

run into the same issue using langchain Runnable method ainvoke. This worked for me but don't know why:

loop = asyncio.get_running_loop() loop.set_task_factory(None)

This Worked for Me! 🚀

I was facing an issue where my Azure Function using LangChain's ainvoke() would only process the first request. After that, it would stop accepting new requests. I also saw this error:

TypeError: Dispatcher.dispatch_forever.<locals>.<lambda>() got an unexpected keyword argument 'context'

After some debugging, I realized that resetting the task factory multiple times was causing issues. Initially, I was doing:

loop = asyncio.get_running_loop()
loop.set_task_factory(None)

But this was being called on every request, which was breaking things.

How I Fixed It

I wrapped the task factory reset in a context manager, so it only resets during execution and restores the original factory after:

import asyncio

class ResetTaskFactory:
    def __enter__(self):
        self.loop = asyncio.get_running_loop()
        self.old_factory = self.loop.get_task_factory()
        self.loop.set_task_factory(None)

    def __exit__(self, exc_type, exc_value, traceback):
        self.loop.set_task_factory(self.old_factory)

# ✅ Usage inside Azure Function
async def main(req):
    with ResetTaskFactory():  # Ensures proper reset
        response = await tool.ainvoke("Hello!")
    return response

Why This Works?

✅ Prevents multiple resets that break async execution.
✅ Restores the original task factory after execution.
✅ Works properly with Azure Functions + LangChain/LangGraph.

Hope this helps someone! 🚀 Let me know if you have a better approach! 😊

Gautamvaishnav-git avatar Mar 25 '25 21:03 Gautamvaishnav-git

Has there been any word on the release for this? Is it now out or still being tested?

jreed-Aces avatar May 27 '25 17:05 jreed-Aces

@jreed-Aces, this has been included in runtime versions 4.1037+. Are you still facing issues?

hallvictoria avatar May 27 '25 18:05 hallvictoria

We were holding off updating until we saw an official release mentioned here. Thank you!

jreed-Aces avatar May 29 '25 19:05 jreed-Aces