pytest-asyncio
pytest-asyncio copied to clipboard
pytest-dev
Async test function gets skipped with `asyncio_mode = auto` and `@unittest.mock.patch.dict` decorator
Hi!
In a PR for a project I'm working on, an async test method gets skipped with a PytestUnhandledCoroutineWarning even though pytest-asyncio is installed and asyncio_mode is auto. I poked at it for while and found that this is because the method was decorated with @unittest.mock.patch.dict(...). Here's a minimal reproducer which exposes the issue with Python 3.10.6, pytest-7.1.2, pytest-asyncio-0.19.0:
--- 8< --- test_the_things.py ---
from unittest import mock
import pytest
bar = {}
async def test_runs():
pass
@mock.patch.dict("test_the_things.bar")
async def test_doesnt_run():
pass
@pytest.mark.asyncio
@mock.patch.dict("test_the_things.bar")
async def test_runs_anyway():
pass
--- >8 ---
Here's the output I get:
(pytest-asyncio-issue) nils@makake:~/test/python/pytest-asyncio> pytest --asyncio-mode=auto -v
======================================== test session starts =========================================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0 -- /home/nils/.virtualenvs/pytest-asyncio-issue/bin/python
cachedir: .pytest_cache
rootdir: /home/nils/test/python/pytest-asyncio
plugins: asyncio-0.19.0
asyncio: mode=auto
collected 3 items
test_the_things.py::test_runs PASSED [ 33%]
test_the_things.py::test_doesnt_run SKIPPED (async def function and no async plugin instal...) [ 66%]
test_the_things.py::test_runs_anyway PASSED [100%]
========================================== warnings summary ==========================================
test_the_things.py::test_doesnt_run
/home/nils/.virtualenvs/pytest-asyncio-issue/lib/python3.10/site-packages/_pytest/python.py:181: PytestUnhandledCoroutineWarning: async def functions are not natively supported and have been skipped.
You need to install a suitable plugin for your async framework, for example:
- anyio
- pytest-asyncio
- pytest-tornasync
- pytest-trio
- pytest-twisted
warnings.warn(PytestUnhandledCoroutineWarning(msg.format(nodeid)))
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================== 2 passed, 1 skipped, 1 warning in 0.01s ===============================
sys:1: RuntimeWarning: coroutine 'test_doesnt_run' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
(pytest-asyncio-issue) nils@makake:~/test/python/pytest-asyncio>
Either without the decorator or by explicitly adding @pytest.mark.asyncio the async test functions don't get skipped.
This also happens when other marks are applied.
@pytest.mark.integration
async def test_runs():
...
The test gets skipped with the same warning listed above when running pytest -m integraton
As above applying the decorator fixes it:
@pytest.mark.integration
@pytest.mark.asyncio
async def test_runs():
...
On closer inspection after looking at this issue I see our tests that are skipped are also marked with a patch @mock.patch("patched.function"). We found a few marked as @pytest.mark.integration that didn't get skipped so it seems as though it is specific to patching.
When I did a test a couple of days ago I could reproduce the issue with @mock.patch.dict, but not @mock.patch. I found that patch performs some special treatment for coroutines, which patch.dict does not.
In my opinion there are two things to be done here:
- Investigate if this is a CPython bug
- Find an intermediate solution for pytest-asyncio
In our code base it has been hard for me to track down the exact cause because it has shown up in some places I didn't expect and not in some I did. I know we do have @mock.patch.dict in places so that could be what is happening.
I looked into it some with @mock.patch.dict and saw the function no longer gets reported as an async function with inspect which seems to match up with what you found.
I opened a CPython issue regarding this: https://github.com/python/cpython/issues/98086
The issue has been resolved for CPython 3.12 and the patch was backported to CPython 3.10 and CPython 3.11.
3.10.9 is scheduled for 2022-12-05 and 3.11.1 should be released in December as well.
CPython 3.7 is not affected by this bug, but the question is if we want to include a workaround for 3.8 and 3.9.
The issue continues to be valid. If anyone wants to resolve it for Python 3.8 or 3.9, they're welcome to provide a backport that can be included in pytest-asyncio.