aiomysql icon indicating copy to clipboard operation
aiomysql copied to clipboard

Sometimes it broke by concurrent.futures._base.CancelledError

Open yanjieee opened this issue 3 years ago • 5 comments

When I using Aiomysql in a sanic high concurrency web server:

async with self._pool.acquire() as conn:
    async with conn.cursor() as cur:
        await cur.execute(query, param)
        if is_all:
            res = await cur.fetchall()
        else:
            res = await cur.fetchone()

Sometimes it broken this error:

Traceback (most recent call last):
  File "/usr/local/mycodes/amysql.py", line 249, in query
    res = await cur.fetchone()
  File "/usr/local/lib/python3.6/site-packages/aiomysql/utils.py", line 103, in __aexit__
    await self._pool.release(self._conn)
concurrent.futures._base.CancelledError

or

Traceback (most recent call last):
  File "/usr/local/mycodes/amysql.py", line 243, in query
    async with self._pool.acquire() as conn:
  File "/usr/local/lib/python3.6/site-packages/aiomysql/utils.py", line 98, in __aenter__
    self._conn = await self._coro
  File "/usr/local/lib/python3.6/site-packages/aiomysql/pool.py", line 133, in _acquire
    async with self._cond:
  File "/usr/lib64/python3.6/asyncio/locks.py", line 79, in __aenter__
    yield from self.acquire()
  File "/usr/lib64/python3.6/asyncio/locks.py", line 181, in acquire
    yield from fut
concurrent.futures._base.CancelledError

About 1000 request in 10 seconds will cause this error. What can I do for avoiding this error?

yanjieee avatar Aug 09 '21 02:08 yanjieee

I got the same error and I don‘t how to solve

stolenzc avatar Nov 02 '21 12:11 stolenzc

I got the same error and I don‘t how to solve

实际上好像并没有什么影响,我现在这么处理:

        except asyncio.CancelledError:
            # 在高并发情况下,偶发该错误,但是res结果并无影响
            # google上建议把协程的CancelledError抛到上级协程一般处理
            self._log.warning("Concurrent CancelledError:{}".format(sql))
            raise

yanjieee avatar Nov 03 '21 02:11 yanjieee

async def query_one(sql, args=(), size=None):
    async with _mysql_pool.acquire() as conn:
        try:
            await conn.ping(reconnect=True)
        except Exception:
            await conn._connect()
        async with conn.cursor(aiomysql.DictCursor) as cur:
            try:
                await cur.execute(sql, args)
                query = await cur.fetchone()
               # 通常在此之前抛出异常。导致事务未能提交,数据库会产生堆积sleep任务等待事务提交
            finally:
                await conn.commit()
            return query

我现在是这样一个函数,即使捕捉到了异常,也不能将此次事务查询完成

stolenzc avatar Nov 03 '21 02:11 stolenzc

I got the same error and I don‘t how to solve.

flask-rabmq avatar Dec 27 '22 06:12 flask-rabmq

async def query_one(sql, args=(), size=None):
    async with _mysql_engine.acquire() as conn:
        
        trans = await conn.begin()
        
        try:
            await conn.execute(sql, args)
        except asyncio.CancelledError:
            # coroutine is canceled ,how to release the transaction?
            # will raise: Cannot release a connection with not finished transaction
            await trans.rollback()
        finally:
            await conn.commit()


ShownYoung avatar Apr 12 '23 02:04 ShownYoung