fastapi
fastapi copied to clipboard
tiangolo
Execute dependencies in parallel instead of sequentially awaiting them
Is your feature request related to a problem? Please describe. FastAPI's Dependency injection system, while extremely useful, tends to run into a problem when several long-running dependencies are needed for a single route, such as with the example below.
from fastapi import Depends, FastAPI
from asyncio import sleep as async_sleep
app = FastAPI()
async def get_big_data():
await async_sleep(5)
return "𝐃𝐀𝐓𝐀"
async def compute_pi():
await async_sleep(5)
return "About 3, I think?"
async def do_nothing_for_a_bit():
await async_sleep(5)
return "Gotta let the CPU spin down for a bit, otherwise it could explode."
@app.get("/do_long_thing")
async def do_long_thing(
big_dep1: str = Depends(get_big_data),
big_dep2: str = Depends(compute_pi),
big_dep3: str = Depends(do_nothing_for_a_bit)
):
return {
'big': big_dep1,
'pi': big_dep2,
'sleep': big_dep3,
}
Running that application and thentrying to access /do_long_thing takes 15 seconds to load, even though none of those dependencies are reliant on each other and could very well be ran in parallel.
Describe the solution you'd like Don't await coroutines and threadpools here. Collect them into a list instead and gather them.
https://github.com/tiangolo/fastapi/blob/3f9f4a0f8f7f5db50fbdc3157fe692cf4e68e13f/fastapi/dependencies/utils.py#L442-L456
Describe alternatives you've considered Asking that the user take care of running and gathering those dependencies themselves is pretty much the only solution to this problem available right now (that I can see, anyway, there might be another one), which undermines the usefulness of the dependency injection system.
This seems like a good idea. But I think it would be quite non-trivial to implement. It would require sorting and finding the dependencies before running them, to be able to know what can be parallelizable that doesn't require something else first.
But if you feel like trying to take a stab at it, a PR would be more than welcome. :muscle:
Hey @tiangolo , just a heads up it looks like @ariloulaleelay has a PR up to implement this in #3902, and I've also worked out how to do it in di (which is based on FastAPI's DI system), so there's at least a couple ideas on how to move forward with this 😄
I'm wondering what's up with this issue. I noticed that PR #3902 for it was closed by the PR's author in May 2022. I've been investigating an overhead of 30ms-100ms depending on hardware performance, wondering if this could get that overhead down.
