asyncpg
asyncpg copied to clipboard
Asyncpg does not seem to clear memory allocated to Pool object after db.disconnect() (possible memory leak)
- asyncpg version: 0.25.0 (tried with 0.23.0 and 0.24.0)
- PostgreSQL version: 11.16
- Do you use a PostgreSQL SaaS? If so, which? Can you reproduce the issue with a local PostgreSQL install?: No, using the postgres:11.16 docker image directrly
- Python version: 3.10.5 (+ latest 3.9 and 3.8)
- Platform: Ubuntu 64 and docker
- Do you use pgbouncer?: No
- Did you install asyncpg with pip?: Yes
- If you built asyncpg locally, which version of Cython did you use?: N/A
- Can the issue be reproduced under both asyncio and uvloop?: I only tried asyncio
We observed PODs running out of memory. They are reading from a queue constantly and calling this function
async def process_request(message, db : ):
await db.connect()
# do_something(message, db)
await db.disconnect()
I am confident this is related to asyncpg as here are some of the objects that are created in memory with every iteration of connect/disconnect.
asyncpg.pgproto.pgproto.ReadBuffer | 1000 | 132.81 KB
collections.OrderedDict | 1000 | 125.00 KB
asyncpg.pool.PoolConnectionHolder | 1000 | 117.19 KB
asyncio.events.TimerHandle | 1001 | 109.48 KB
I have tried:
- waiting for the TimerHandle to expire (60s)
- calling engine.dispose() manually
- calling the garbage collector manually
Sometime in the past we encountered a problem that made 10000s of connections in a few seconds (crashing the DB) that was fixed by abandoning this connect/disconnect pattern, so maybe these two are related.
The db is initialised as such
databases.Database(db_uri)
This problem does not exists in the sqlite driver.
Can you reproduce this by using just asyncpg
? Otherwise it isn't clear if this isn't a bug in databases
.