omnilib
Aiosqlite 0.22.0 hangs under SqlAlchemy
We are running "canary" tests in Airflow and our regular tests started to hang around 13 hours ago - in a very weird way - all tests pass when we are running it with sqlite, but the whole pytest session hangs. Normally those tests complete in about 27 minutes, but the pytest call started to hang for another 30 minutes and then out built-in timeouts kill it. But there is no log nor indication what could go wrong.
After doing a bit of bisecting and seing what changed I arrived to conclusion that it's the aiosqlite bumped from 0.21.0 to 0.22.0 caused it - this is the only difference we can see between successfull and failed run that could have caused it.
My best guess is that this is some side-effect of https://github.com/omnilib/aiosqlite/pull/305 -> this looks like some connection being waited on is lost (or similar issue).
Some more info:
Example successful run: https://github.com/apache/airflow/actions/runs/20194148497/job/57981065405 - there we upgraded deps but not aiosqlite (0.22.0 was not released yet): https://github.com/apache/airflow/actions/runs/20194148497/job/57981065457#step:5:1351
The next run that failed with the timeout was: https://github.com/apache/airflow/actions/runs/20196805596/job/57982265834 -> you can see it's 1hr and killed. This one had iosqlite 0.22.0 already: https://github.com/apache/airflow/actions/runs/20196805596/job/57982265781#step:5:1340
I am going to double check my hypothesis with limiting aiosqlite to != 0.22.0 and will confirm it, but I am almost 100% sure there is an issue here.
The https://github.com/apache/airflow/pull/59406 is going to give us ultimate answer
Yep. Confirmed. Just limiting aiosqlite to !=0.22.0 fixes the problem of pytest session hanging for us.
Can confirm this is also happening for us, and adding the aiosqlite<0.22 constraint fixed it.
For reference, the list of our 3rd-party dependencies for the environment where the pytest hanging issue occurs:
aiosqlite==0.22.0
alembic==1.16.2
amqpstorm==2.11.1
annotated-types==0.7.0
anyio==4.9.0
asgi-correlation-id==4.3.4
asyncio==3.4.3
asyncmy==0.2.10
attrs==25.3.0
cattrs==25.1.0
certifi==2025.4.26
cffi==1.17.1
charset-normalizer==3.4.2
click==8.2.1
clickclick==20.10.2
connexion==2.14.2
coverage==7.8.2
cryptography==45.0.3
deepdiff==8.5.0
dnspython==2.7.0
docker==7.1.0
docstring-to-markdown==0.17
dramatiq==1.18.0
elastic-transport==8.17.1
elasticsearch==9.0.1
email-validator==2.2.0
fastapi==0.115.12
fastapi-cli==0.0.7
fernet==1.0.1
flake8==7.2.0
flask==2.2.5
flask-cors==6.0.0
gevent==25.5.1
globus-sdk==3.56.1
greenlet==3.2.2
gunicorn==23.0.0
h11==0.16.0
h2==4.2.0
hpack==4.1.0
httpcore==1.0.9
httptools==0.6.4
httpx==0.28.1
hypercorn==0.17.3
hyperframe==6.1.0
idna==3.10
importlib-metadata==8.7.0
inflection==0.5.1
iniconfig==2.1.0
itsdangerous==2.2.0
jedi==0.19.2
jinja2==3.1.6
jsonschema==4.24.0
jsonschema-specifications==2025.4.1
lsprotocol==2023.0.1
mako==1.3.10
markdown-it-py==3.0.0
markupsafe==3.0.2
mccabe==0.7.0
mdurl==0.1.2
mypy==1.16.0
mypy-extensions==1.1.0
mysql-connector-python==9.3.0
numpy==2.2.6
orderly-set==5.4.1
outcome==1.3.0.post0
packaging==25.0
pamqp==2.3.0
pandas==2.2.3
parso==0.8.4
pathspec==0.12.1
pendulum==3.1.0
pika==1.3.2
platformdirs==4.3.8
pluggy==1.6.0
priority==2.0.0
prometheus-client==0.22.1
pyaes==1.6.1
pycodestyle==2.13.0
pycparser==2.22
pydantic==2.11.5
pydantic-core==2.33.2
pydantic-settings==2.9.1
pyflakes==3.3.2
pygments==2.19.1
pyjwt==2.10.1
pylsp-mypy==0.7.0
pylsp-rope==0.1.17
pytest==8.4.0
pytest-asyncio==1.0.0
pytest-cov==6.1.1
pytest-flask==1.3.0
pytest-integration-mark==0.2.0
pytest-mock==3.14.1
pytest-sqlalchemy-mock==0.1.7
python-dateutil==2.9.0.post0
python-dotenv==1.1.0
python-json-logger==3.3.0
python-lsp-jsonrpc==1.1.2
python-lsp-ruff==2.2.2
python-lsp-server==1.12.2
python-multipart==0.0.20
pytoolconfig==1.3.1
pytz==2025.2
pyyaml==6.0.2
referencing==0.36.2
requests==2.32.3
requests-mock==1.12.1
rich==14.0.0
rich-toolkit==0.14.7
rope==1.13.0
rpds-py==0.25.1
ruff==0.11.12
schedule==1.2.2
setuptools==80.9.0
shellingham==1.5.4
six==1.17.0
slack-sdk==3.35.0
sniffio==1.3.1
sortedcontainers==2.4.0
sqlalchemy==2.0.41
starlette==0.46.2
structlog==25.4.0
swagger-ui-bundle==0.0.9
testcontainers==4.10.0
trio==0.30.0
typer==0.16.0
types-requests==2.32.0.20250602
typing-extensions==4.14.0
typing-inspection==0.4.1
tzdata==2025.2
ujson==5.10.0
urllib3==2.4.0
uvicorn==0.34.3
uvloop==0.21.0
watchfiles==1.0.5
websockets==15.0.1
werkzeug==2.2.3
wrapt==1.17.2
wsproto==1.2.0
zipp==3.22.0
zope-event==5.0
zope-interface==7.2
Apologies for the churn this caused. I will look into it and root cause which PR triggered this behavior, and then see if I can work out a fix without needing to revert the changes.
At least for cocat, I'm pretty sure this is caused by SqlAlchemy not closing the aiosqlite connection, and instead relying on setting connection.daemon = True with the expectation that the lingering thread(s) will magically disappear, with no regard for the state of the connection when that happens.
I confirmed with extra debug logging that at no point does sqlalchemy call/await aiosqlite's .close() method. I've also confirmed that with aiosqlite v0.22.0, changing that line in sqlalchemy to connection._thread.daemon = True prevents the hang in cocat's test suite.
"you'll thank us later" indeed.
I confirmed with extra debug logging that at no point does sqlalchemy call/await aiosqlite's .close() method. I've also confirmed that with aiosqlite v0.22.0, changing that line in sqlalchemy to connection._thread.daemon = True prevents the hang in cocat's test suite.
Nice
Coming from the SQLAlchemy issue
SQLAlchemy uses connection pooling with aiosqlite by default, so the behaviour (close is not called) is expected, at least in an example like the one reported in the sqlalchemy issue https://github.com/sqlalchemy/sqlalchemy/issues/13039.
As mentioned there the alternative are:
- use null pool to disable connection pooling
- dispose the engine before existing the loop when using connection pooling
If there are other issues encountered while doing either of the above, then please provide an example of such cases in the SQLAlchemy issue.
Hi!
I encountered a similar issue in my project. What I found strange in my CI is that the tests pass on Python 3.11 and 3.13, but fail on 3.12 and 3.14, with the following error:
unraisableexception.py:67: PytestUnraisableExceptionWarning: Exception ignored in: <function Connection.__del__ at 0x7f81bbd319e0>
Traceback (most recent call last):
File "/home/runner/.cache/pypoetry/virtualenvs/fastapi-do-zero-b-ugIaxI-py3.12/lib/python3.12/site-packages/aiosqlite/core.py", line 429, in __del__
self._stop_running()
File "/home/runner/.cache/pypoetry/virtualenvs/fastapi-do-zero-b-ugIaxI-py3.12/lib/python3.12/site-packages/aiosqlite/core.py", line 101, in _stop_running
future = asyncio.get_event_loop().create_future()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.12.12/x64/lib/python3.12/asyncio/events.py", line 702, in get_event_loop
raise RuntimeError('There is no current event loop in thread %r.'
RuntimeError: There is no current event loop in thread 'Dummy-7'.
This issue only appears in Python 3.12 and 3.14, while tests pass on Python 3.11 and 3.13.
For reference, here are the CI logs:
- full ci: https://github.com/dunossauro/fastapi-do-zero/actions/runs/20403385875
- 3.12 (failing): https://github.com/dunossauro/fastapi-do-zero/actions/runs/20403385875/job/58629112770#step:8:365
- 3.14 (failing): https://github.com/dunossauro/fastapi-do-zero/actions/runs/20403385875/job/58629112775#step:8:374
