asyncpg
asyncpg copied to clipboard
MagicStack
Does not work on ASGI servers
- asyncpg version: 0.21.0
- PostgreSQL version: 13
- Do you use a PostgreSQL SaaS? If so, which? Can you reproduce the issue with a local PostgreSQL install?: N/A (No)
- Python version: 3.9
- Platform: Fedora Rawhide
- Do you use pgbouncer?: No
- Did you install asyncpg with pip?: Yes
- If you built asyncpg locally, which version of Cython did you use?:
- Can the issue be reproduced under both asyncio and uvloop?: Yes
When running asyncpg on an ASGI server (FastAPI/Quart), asyncpg crashes with another operation in progress. This does not happen on AIOHTTP or non-ASGI servers
Also, when using uvicorn, i get an error
Traceback (most recent call last):
File "/usr/bin/uvicorn", line 33, in
This is not a dupe of #309 as this occurs on ALL asyncpg calls made in a route and not just when its been silent for too long
asyncpg crashes with another operation in progress
It seems like an error in your app, you're likely attempting to share a single connection across multiple concurrent tasks.
As for the "loop is already running" the error is clear: you're attempting to run loop.run_until_complete() within another loop.run_until_complete(). I'm not very familiar with ASGI, but it seems like you're not supposed to use loop.run_until_complete() in your modules as the whole app is already inside a running loop.
Following on from @elprans - since I got stuck on the same thing.
Try using asyncpg.create_pool() at app startup and storing the result in your global app state.
I do something like app.state.pgpool = asyncpg.create_pool().
Then for each request grab a connection:
try:
conn = await app.state.pgpool.acquire()
# do stuff with conn
finally:
await app.state.pgpool.release(conn)
It solved the problem for me
AsyncPG is doing very strange things to me with FastAPI.
my environment have:
- python 3.8.1
- asyncpg 0.23.0
- postgres 12.7
- fastAPI 0.65.0
I made a wrapper that currently is basically this:
class DataLinker:
def __init__(self, dsn: str):
self._dsn = dsn
self.connection: asyncpg.Connection = None
async def init(self):
self.connection = await asyncpg.connect(dsn=self._dsn)
def __repr__(self):
return f'DataLinker(dsn=postgres://user:password@host:port/database'
async def close(self):
await self.connection.close()
async def execute(self, query, *params):
return await self.connection.execute(query, *params)
the idea is, in the finals projects, having a connectors file where I instance postgre_handler = DataLiker(dsn_str), and on FastAPI startup event preform postgre_handler.init().
At this point, apparently everything is ok. But...
When on model files I call result = await postgre_handler.execute(qry_str, param1, paramN) I get 'coroutine' object is not callable, under the last line of the DataLiker file (the code above). If a change those lines and have
async def execute(self, query, *params):
result = self.connection.execute(query, *params)
return await result
it raise coroutine object argument after * must be an iterable, not coroutine.
if i have:
async def execute(self, query, *params):
result = self.connection.execute(query, *params)
new_result = await result
return new_result
this totally explode, shutdown the interpreter even with all the exception handler to maintain fastAPI working. In the console prints Process finished with exit code -1073741819 (0xC0000005). And this make me think that could be a problem with python itself. particularly for being different if I did return await result or new_result = await result.
The most weird thing to me is that if i left:
async def execute(self, query, *params):
return self.connection.execute(query, *params)
And in the model file i call
result = postgre_handler.execute(qry_str, param1, paramN)
awaited_result = await result
it works....
idk what is happening.. im trying to change versions of asyncpg and python, and see what happens.
Maybe good to close this issue, as asyncpg does work fine with ASGI, e.g. gunicorn & uvicorn and fastapi, using the techniques described by others here earlier.
