prisma-client-py icon indicating copy to clipboard operation
prisma-client-py copied to clipboard

when the main app is shutdown, prisma rust is still keeping the connection open in the background

Open vikyw89 opened this issue 1 year ago • 3 comments

Bug description

image and .env changes won't get reflected in the client (example changing url param, connection_limit, etc) which I think is caused by the old prisma rust is still active and using the old .env image

How to reproduce

There are 2 ways:

  1. try to exhaust all connection pool in the db, and kill the client / port. Then try connecting again with less number of connection pool, it will raise this error
prisma.errors.DataError: Too many database connections opened: ERROR HY000 (1040): Too many connections```

2. run client, then kill server, try to change .env param and run again, client will still use the old .env param
<!--
Steps to reproduce the behavior:
1. Go to '...'
2. Change '....'
3. Run '....'
4. See error
-->

## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

prisma rust should shutdown when the main python prisma client is shutdown

## Prisma information

<!-- Your Prisma schema, Prisma Client Python queries, ...
Do not include your database credentials when sharing your Prisma schema! -->
latest as of 20240806


```prisma

Environment & setup

  • OS: ubuntu24
  • Database: mysql
  • Python version: 12
  • Prisma version: latest as of today 20240806
prisma client python    : 0.14.0
platform                : debian-openssl-3.0.x
expected engine version : 393aa359c9ad4a4bb28630fb5613f9c281cde053
installed extras        : []
install path            : /home/vikyw/.cache/pypoetry/virtualenvs/benchmark-cb5ao4-8-py3.12/lib/python3.12/site-packages/prisma
binary cache dir        : /home/vikyw/.cache/prisma-python/binaries/5.17.0/393aa359c9ad4a4bb28630fb5613f9c281cde053

vikyw89 avatar Aug 06 '24 08:08 vikyw89

@vikyw89 I can't reproduce this with mysql v8, could you share a script to reproduce the issue?

My script
import asyncio

from prisma import Prisma


async def check_mysql_connections():
    prisma = Prisma()
    await prisma.connect()

    # Get total number of connections
    total_conn = await prisma.query_raw("SHOW STATUS WHERE Variable_name = 'Threads_connected';")
    print(f"Total connections: {total_conn[0]['Value']}")

    # Get max allowed connections
    max_conn = await prisma.query_raw("SHOW VARIABLES LIKE 'max_connections';")
    print(f"Max connections allowed: {max_conn[0]['Value']}")

    # Get detailed information about current connections
    detailed = await prisma.query_raw("""
        SELECT 
            ID,
            USER,
            HOST,
            DB,
            COMMAND,
            TIME,
            STATE,
            INFO
        FROM 
            INFORMATION_SCHEMA.PROCESSLIST
        ORDER BY 
            TIME DESC
        LIMIT 10;
    """)
    print('\nDetailed connection information (latest 10):')
    for row in detailed:
        print(row)

    # Get connection counts by user and host
    user_host_counts = await prisma.query_raw("""
        SELECT 
            USER,
            HOST,
            COUNT(*) AS connection_count
        FROM 
            INFORMATION_SCHEMA.PROCESSLIST
        GROUP BY 
            USER, HOST
        ORDER BY 
            connection_count DESC;
    """)
    print('\nConnection counts by user and host:')
    for row in user_host_counts:
        print(f"{row['USER']}@{row['HOST']}: {row['connection_count']}")

    # Get connection counts by state
    state_counts = await prisma.query_raw("""
        SELECT 
            STATE,
            COUNT(*) AS connection_count
        FROM 
            INFORMATION_SCHEMA.PROCESSLIST
        GROUP BY 
            STATE
        ORDER BY 
            connection_count DESC;
    """)
    print('\nConnection counts by state:')
    for row in state_counts:
        state = row['STATE'] if row['STATE'] else 'NULL'
        print(f"{state}: {row['connection_count']}")


asyncio.run(check_mysql_connections())

Running this script always outputs the same information for me, I'd expect it to be different if the connections weren't actually being closed.

RobertCraigie avatar Aug 18 '24 16:08 RobertCraigie

https://github.com/user-attachments/assets/46043c7f-4ba4-43bb-84f4-f12856836fa2

vikyw89 avatar Aug 27 '24 19:08 vikyw89

poetry run test_db.py script

def run():
    import asyncio
    
    asyncio.run(arun())
    
async def arun():

    # await check_mysql_connections()

    await check_postgresql_connections()
    
async def check_mysql_connections():
    prisma = Prisma()
    await prisma.connect()

    # Get total number of connections
    total_conn = await prisma.query_raw("SHOW STATUS WHERE Variable_name = 'Threads_connected';")
    print(f"Total connections: {total_conn[0]['Value']}")

    # Get max allowed connections
    max_conn = await prisma.query_raw("SHOW VARIABLES LIKE 'max_connections';")
    print(f"Max connections allowed: {max_conn[0]['Value']}")

    # Get detailed information about current connections
    detailed = await prisma.query_raw("""
        SELECT 
            ID,
            USER,
            HOST,
            DB,
            COMMAND,
            TIME,
            STATE,
            INFO
        FROM 
            INFORMATION_SCHEMA.PROCESSLIST
        ORDER BY 
            TIME DESC
        LIMIT 10;
    """)
    print('\nDetailed connection information (latest 10):')
    for row in detailed:
        print(row)

    # Get connection counts by user and host
    user_host_counts = await prisma.query_raw("""
        SELECT 
            USER,
            HOST,
            COUNT(*) AS connection_count
        FROM 
            INFORMATION_SCHEMA.PROCESSLIST
        GROUP BY 
            USER, HOST
        ORDER BY 
            connection_count DESC;
    """)
    print('\nConnection counts by user and host:')
    for row in user_host_counts:
        print(f"{row['USER']}@{row['HOST']}: {row['connection_count']}")

    # Get connection counts by state
    state_counts = await prisma.query_raw("""
        SELECT 
            STATE,
            COUNT(*) AS connection_count
        FROM 
            INFORMATION_SCHEMA.PROCESSLIST
        GROUP BY 
            STATE
        ORDER BY 
            connection_count DESC;
    """)
    print('\nConnection counts by state:')
    for row in state_counts:
        state = row['STATE'] if row['STATE'] else 'NULL'
        print(f"{state}: {row['connection_count']}")


async def check_postgresql_connections():
    prisma = Prisma()
    await prisma.connect()

    # Get total number of connections
    total_conn = await prisma.query_raw("""
        SELECT COUNT(*)
        FROM pg_stat_activity;
    """)
    print(f"Total connections: {total_conn[0]['count']}")

    # Get max allowed connections
    max_conn = await prisma.query_raw("""
        SELECT setting::int
        FROM pg_settings
        WHERE name = 'max_connections';
    """)
    print(f"Max connections allowed: {max_conn[0]['setting']}")

    # Get detailed information about current connections
    detailed = await prisma.query_raw("""
        SELECT 
            pid,
            usename,
            client_addr,
            datname,
            state,
            query,
            backend_start,
            query_start
        FROM 
            pg_stat_activity
        ORDER BY 
            query_start DESC
        LIMIT 10;
    """)
    print('\nDetailed connection information (latest 10):')
    for row in detailed:
        print(row)

    # Get connection counts by user and host
    user_host_counts = await prisma.query_raw("""
        SELECT 
            usename,
            client_addr,
            COUNT(*) AS connection_count
        FROM 
            pg_stat_activity
        GROUP BY 
            usename, client_addr
        ORDER BY 
            connection_count DESC;
    """)
    print('\nConnection counts by user and host:')
    for row in user_host_counts:
        print(f"{row['usename']}@{row['client_addr']}: {row['connection_count']}")

    # Get connection counts by state
    state_counts = await prisma.query_raw("""
        SELECT 
            state,
            COUNT(*) AS connection_count
        FROM 
            pg_stat_activity
        GROUP BY 
            state
        ORDER BY 
            connection_count DESC;
    """)
    print('\nConnection counts by state:')
    for row in state_counts:
        state = row['state'] if row['state'] else 'NULL'
        print(f"{state}: {row['connection_count']}")

    await prisma.disconnect()

poetry run start script

def run():
    import subprocess
    import multiprocessing
    
    cpu_count = multiprocessing.cpu_count()
    subprocess.run("poetry run prisma migrate deploy", shell=True)
    subprocess.run(f"poetry run uvicorn server.main:app --host 0.0.0.0 --port 8000 --workers {20}", shell=True)

this is my DATABASE_URL

DATABASE_URL=postgres://postgres:postgres@localhost:5432/postgres

this is my prisma init script

prisma = Prisma(auto_register=True, http=HttpConfig(timeout=Timeout(None,pool=None)))

I'm using fastapi server

vikyw89 avatar Aug 27 '24 19:08 vikyw89