peewee-async
peewee-async copied to clipboard
something wrong in my app with error "close cannot be used while an asynchronous query is underway"
Renctly I tried to redeploy my project used with sanic frame ,some errors happened when I do a test under high-concurrency. info in the log:
close cannot be used while an asynchronous query is underway
and the traceback is below:
Traceback (most recent call last):
File "/home/mwh/ad_sanic/handlers/oprs.py", line 17, in post
_parsed_req = await ReqParser(_req_body,'parse',req_ip=(request.headers['Remote-Addr']).split(':')[0]).parser()
File "/home/mwh/ad_sanic/utils/req_parse.py", line 49, in parser
_mapped['sim_country_iso'] = await self.geo()
File "/home/mwh/ad_sanic/utils/req_parse.py", line 75, in geo
db_country = (await self.db.get(DbIp.select(DbIp.country).where((DbIp.ip_from<=self._req_ip)&(DbIp.ip_to>=self._req_ip)))).country
File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 169, in get
result = yield from self.execute(query.limit(1))
File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 269, in execute
return (yield from execute(query))
File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 428, in execute
return (yield from coroutine(query))
File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 569, in select
cursor = yield from _execute_query_async(query)
File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 1491, in _execute_query_async
return (yield from _run_sql(query.database, *query.sql()))
File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 1476, in _run_sql
cursor = yield from database.cursor_async()
File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 902, in cursor_async
return (yield from self._async_conn.cursor(conn=conn))
File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 1067, in cursor
conn = yield from self.acquire()
File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 1036, in acquire
return (yield from self.pool.acquire())
File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/aiopg/utils.py", line 67, in __iter__
resp = yield from self._coro
File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/aiopg/pool.py", line 180, in _acquire
yield from self._cond.wait()
File "/usr/local/lib/python3.5/asyncio/locks.py", line 326, in wait
yield from fut
File "/usr/local/lib/python3.5/asyncio/futures.py", line 385, in __iter__
yield self # This tells Task to wait for completion.
File "uvloop/future.pyx", line 434, in uvloop.loop.BaseTask._fast_wakeup (uvloop/loop.c:112710)
File "/usr/local/lib/python3.5/asyncio/futures.py", line 266, in result
raise CancelledError
concurrent.futures._base.CancelledError
what caused this problem?
Probably that's some kind of edge case with uvloop
and aiopg
. Does this error happen without uvloop
?
I have this bug without uvloop.
Yes. I have the same issue. I wanna testing my model with aiohttp and unit test. I would try to solve it for an other driver, local threads parameters or override event loop. But looks like its not helpful, manage connections has no effect too. It's just testing, what if that issue will happen in a production.
You should explicity call objects.close()
before close event loop
I get this while performing unit tests.
I have a series of simple tests to grab an entity with a simple manager context query. If I try to execute more than one during a test, I get this error.
_common/common/db/getters.py:82: in get_generic
return await entity_getter(node_tag=node_tag, client_code=client_code)
_common/common/db/getters.py:17: in get_store
Client.code ** client_code
/home/adam/.virtualenvs/mxtest/lib/python3.6/site-packages/peewee_async.py:169: in get
result = yield from self.execute(query)
/home/adam/.virtualenvs/mxtest/lib/python3.6/site-packages/peewee_async.py:269: in execute
return (yield from execute(query))
/home/adam/.virtualenvs/mxtest/lib/python3.6/site-packages/peewee_async.py:433: in execute
return (yield from coroutine(query))
/home/adam/.virtualenvs/mxtest/lib/python3.6/site-packages/peewee_async.py:574: in select
cursor = yield from _execute_query_async(query)
/home/adam/.virtualenvs/mxtest/lib/python3.6/site-packages/peewee_async.py:1555: in _execute_query_async
return (yield from _run_sql(query.database, *query.sql()))
/home/adam/.virtualenvs/mxtest/lib/python3.6/site-packages/peewee_async.py:1542: in _run_sql
return cursor
/home/adam/.virtualenvs/mxtest/lib/python3.6/site-packages/peewee.py:3656: in __exit__
reraise(new_type, new_type(*exc_args), traceback)
/home/adam/.virtualenvs/mxtest/lib/python3.6/site-packages/peewee.py:135: in reraise
raise value.with_traceback(tb)
/home/adam/.virtualenvs/mxtest/lib/python3.6/site-packages/peewee_async.py:1539: in _run_sql
yield from cursor.release
/usr/lib/python3.6/asyncio/coroutines.py:210: in coro
res = func(*args, **kw)
/home/adam/.virtualenvs/mxtest/lib/python3.6/site-packages/peewee_async.py:1107: in release_cursor
cursor.close()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <aiopg.cursor.Cursor object at 0x7fc48e910fd0>
def close(self):
"""Close the cursor now."""
> self._impl.close()
E peewee.ProgrammingError: close cannot be used while an asynchronous query is underway
/home/adam/.virtualenvs/mxtest/lib/python3.6/site-packages/aiopg/cursor.py:51: ProgrammingError
I was getting this same issue.
peewee.ProgrammingError: close cannot be used while an asynchronous query is underway
After tracing through the issue, I determined it was due to my setup with pytest-asyncio since that library does the following in the default event_loop fixture:
Creates and injects a new instance of the default asyncio event loop. By default, the loop will be closed at the end of the test (i.e. the default fixture scope is function).
To fix the issue, I created a conftest.py file at the root of my tests that has the following replacement fixture.
import pytest
import asyncio
@pytest.fixture(scope='session')
def event_loop():
return asyncio.new_event_loop()
Maybe this will help someone else who lands here from google.