typeshed Convert AbstractContextManager from a protocol to an ABC

AbstractContextManager should not be a protocol or else the exit method, which is abstract, cannot be determined by type checkers to be implemented. This prevents users from calling super. For details, see https://github.com/microsoft/pyright/issues/6965#issuecomment-1889704569 and https://github.com/microsoft/pyright/issues/6965

Jan 12 '24 09:01 NeilGirdhar

This decorator also caused a lot of confusion here: https://github.com/pylint-dev/pylint/issues/1594

Jan 12 '24 09:01 NeilGirdhar

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

Jan 12 '24 09:01 github-actions[bot]

See https://discuss.python.org/t/can-we-make-abstractcontextmanager-exit-concrete/43025

Jan 12 '24 10:01 NeilGirdhar

Apart from the necessary stubtest changes (and comment there), could you also add a comment to the stub to explain the situation for posterity?

Jan 12 '24 10:01 srittau

Happy to do that, but there is some debate as to whether this is the right change, so I'll just wait until it's resolved.

Jan 12 '24 11:01 NeilGirdhar

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

Jan 12 '24 11:01 github-actions[bot]

Diff from mypy_primer, showing the effect of this PR on open source code:

pip (https://github.com/pypa/pip)
+ src/pip/_internal/utils/temp_dir.py:147: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TempDirectory"; expected "AbstractContextManager[Never]"  [arg-type]
+ src/pip/_internal/operations/build/build_tracker.py:46: error: Never has no attribute "path"  [attr-defined]
+ src/pip/_internal/operations/build/build_tracker.py:46: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TempDirectory"; expected "AbstractContextManager[Never]"  [arg-type]
+ src/pip/_internal/cli/req_command.py:95: error: Argument 1 to "enter_context" of "CommandContextMixIn" has incompatible type "PipSession"; expected "AbstractContextManager[Never]"  [arg-type]
+ src/pip/_internal/commands/install.py:320: error: Argument 1 to "enter_context" of "CommandContextMixIn" has incompatible type "TempDirectory"; expected "AbstractContextManager[Never]"  [arg-type]

prefect (https://github.com/PrefectHQ/prefect)
+ src/prefect/engine.py:1765: error: Argument 1 to "enter_async_context" of "AsyncExitStack" has incompatible type "PrefectClient"; expected "AbstractAsyncContextManager[PrefectClient]"  [arg-type]
+ src/prefect/engine.py:1765: error: Argument 1 to "enter_async_context" of "AsyncExitStack" has incompatible type "PrefectClient"; expected "AbstractAsyncContextManager[PrefectClient]"  [arg-type]

trio (https://github.com/python-trio/trio)
+ src/trio/_core/_run.py:1015: error: Incompatible return value type (got "NurseryManager", expected "AbstractAsyncContextManager[Nursery]")  [return-value]
+ src/trio/_tests/test_timeouts.py:130: error: "object" has no attribute "__enter__"  [attr-defined]
+ src/trio/_tests/test_timeouts.py:130: error: "object" has no attribute "__exit__"  [attr-defined]
+ src/trio/_core/_tests/test_run.py:735: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "CancelScope"; expected "AbstractContextManager[Never]"  [arg-type]
+ src/trio/_core/_tests/test_run.py:754: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "CancelScope"; expected "AbstractContextManager[Never]"  [arg-type]
+ src/trio/_core/_tests/test_run.py:2360: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "CancelScope"; expected "AbstractContextManager[Never]"  [arg-type]
+ src/trio/_core/_tests/test_run.py:2362: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "CancelScope"; expected "AbstractContextManager[Never]"  [arg-type]

tornado (https://github.com/tornadoweb/tornado)
+ tornado/template.py:752: error: Incompatible return value type (got "Indenter", expected "AbstractContextManager[Any]")  [return-value]
+ tornado/template.py:765: error: Incompatible return value type (got "IncludeTemplate", expected "AbstractContextManager[Any]")  [return-value]

pylint (https://github.com/pycqa/pylint)
+ pylint/lint/pylinter.py:431: error: Need type annotation for "output_file"  [var-annotated]
+ pylint/lint/pylinter.py:432: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TextIOWrapper"; expected "AbstractContextManager[Never]"  [arg-type]

anyio (https://github.com/agronholm/anyio)
+ src/anyio/abc/_sockets.py:145: error: Argument 1 to "enter_async_context" of "AsyncExitStack" has incompatible type "TaskGroup"; expected "AbstractAsyncContextManager[TaskGroup | None]"  [arg-type]
+ src/anyio/abc/_sockets.py:149: error: Item "None" of "TaskGroup | None" has no attribute "start_soon"  [union-attr]
+ src/anyio/pytest_plugin.py:45: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TestRunner"; expected "AbstractContextManager[Never]"  [arg-type]
+ src/anyio/_backends/_trio.py:1108: error: Incompatible return value type (got "_SignalReceiver", expected "AbstractContextManager[AsyncIterator[Signals]]")  [return-value]
+ src/anyio/_backends/_asyncio.py:2437: error: Incompatible return value type (got "_SignalReceiver", expected "AbstractContextManager[AsyncIterator[Signals]]")  [return-value]

starlette (https://github.com/encode/starlette)
+ starlette/_utils.py:48: error: All bases of a protocol must be protocols  [misc]
+ starlette/routing.py:658: error: Incompatible types in assignment (expression has type "_DefaultLifespan", variable has type "Union[Callable[[Any], AbstractAsyncContextManager[None]], Callable[[Any], AbstractAsyncContextManager[Mapping[str, Any]]]]")  [assignment]

Tanjun (https://github.com/FasterSpeeding/Tanjun)
+ tanjun/dependencies/data.py:128: error: Incompatible return value type (got "Lock", expected "AbstractAsyncContextManager[Any]")  [return-value]
+ tanjun/dependencies/limiters.py:215: error: Incompatible return value type (got "_CooldownAcquire", expected "AbstractAsyncContextManager[None]")  [return-value]
+ tanjun/dependencies/limiters.py:340: error: Incompatible return value type (got "_ConcurrencyAcquire", expected "AbstractAsyncContextManager[None]")  [return-value]

mkosi (https://github.com/systemd/mkosi)
+ mkosi/qemu.py:637:25: error: Need type annotation for "ovmf_vars"  [var-annotated]
+ mkosi/qemu.py:637:45: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "_TemporaryFileWrapper[bytes]"; expected "AbstractContextManager[Never]"  [arg-type]
+ mkosi/qemu.py:703:21: error: Need type annotation for "f"  [var-annotated]
+ mkosi/qemu.py:703:41: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "_TemporaryFileWrapper[str]"; expected "AbstractContextManager[Never]"  [arg-type]
+ mkosi/qemu.py:747:23: error: Need type annotation for "scratch"  [var-annotated]
+ mkosi/qemu.py:747:43: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "_TemporaryFileWrapper[bytes]"; expected "AbstractContextManager[Never]"  [arg-type]
+ mkosi/qemu.py:808:20: error: Need type annotation for "file"  [var-annotated]
+ mkosi/qemu.py:809:17: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "_TemporaryFileWrapper[bytes]"; expected "AbstractContextManager[Never]"  [arg-type]
+ mkosi/mounts.py:83:49: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TemporaryDirectory[str]"; expected "AbstractContextManager[Never]"  [arg-type]
+ mkosi/mounts.py:89:33: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TemporaryDirectory[str]"; expected "AbstractContextManager[Never]"  [arg-type]
+ mkosi/mounts.py:95:21: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TemporaryDirectory[str]"; expected "AbstractContextManager[Never]"  [arg-type]
+ mkosi/__init__.py:3024:23: error: Need type annotation for "scratch"  [var-annotated]
+ mkosi/__init__.py:3024:43: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TemporaryDirectory[str]"; expected "AbstractContextManager[Never]"  [arg-type]

pandas-stubs (https://github.com/pandas-dev/pandas-stubs)
+ tests/__init__.py:126: error: Incompatible return value type (got "WarningsChecker", expected "AbstractContextManager[Any]")  [return-value]

nionutils (https://github.com/nion-software/nionutils)
+ nion/utils/ReferenceCounting.py:51: error: Incompatible return value type (got "RefContextManager", expected "AbstractContextManager[ReferenceCounted]")  [return-value]
+ nion/utils/ListModel.py:332: error: Incompatible return value type (got "ChangeTracker", expected "AbstractContextManager[ChangeTracker]")  [return-value]
+ nion/utils/ListModel.py:744: error: Incompatible return value type (got "ChangeTracker", expected "AbstractContextManager[ChangeTracker]")  [return-value]
+ nion/utils/Stream.py:61: error: Incompatible return value type (got "RefContextManager", expected "AbstractContextManager[AbstractStream[T]]")  [return-value]

mongo-python-driver (https://github.com/mongodb/mongo-python-driver)
+ pymongo/client_session.py:755: error: Incompatible return value type (got "_TransactionContext", expected "AbstractContextManager[Any]")  [return-value]

urllib3 (https://github.com/urllib3/urllib3)
+ test/with_dummyserver/test_https.py:739: error: Incompatible types in assignment (expression has type "WarningsChecker", variable has type "AbstractContextManager[object]")  [assignment]

streamlit (https://github.com/streamlit/streamlit)
+ lib/tests/streamlit/web/server/server_test.py: note: In member "test_missing_file" of class "SslServerTest":
+ lib/tests/streamlit/web/server/server_test.py:374:23: error: Need type annotation for "tmp_dir"  [var-annotated]
+ lib/tests/streamlit/web/server/server_test.py:374:48: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TemporaryDirectory[str]"; expected "AbstractContextManager[Never]"  [arg-type]
+ lib/tests/streamlit/web/server/server_test.py:389:20: error: Need type annotation for "logs"  [var-annotated]
+ lib/tests/streamlit/web/server/server_test.py:390:17: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "_AssertLogsContext[_LoggingWatcher]"; expected "AbstractContextManager[Never]"  [arg-type]
+ lib/tests/streamlit/web/server/server_test.py: note: In member "test_invalid_file_content" of class "SslServerTest":
+ lib/tests/streamlit/web/server/server_test.py:407:23: error: Need type annotation for "tmp_dir"  [var-annotated]
+ lib/tests/streamlit/web/server/server_test.py:407:48: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TemporaryDirectory[str]"; expected "AbstractContextManager[Never]"  [arg-type]
+ lib/tests/streamlit/web/server/server_test.py:444:20: error: Need type annotation for "logs"  [var-annotated]
+ lib/tests/streamlit/web/server/server_test.py:445:17: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "_AssertLogsContext[_LoggingWatcher]"; expected "AbstractContextManager[Never]"  [arg-type]
+ lib/tests/streamlit/web/cli_test.py: note: In member "test_ssl" of class "HTTPServerIntegrationTest":
+ lib/tests/streamlit/web/cli_test.py:451:24: error: Need type annotation for "tmp_home"  [var-annotated]
+ lib/tests/streamlit/web/cli_test.py:451:49: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TemporaryDirectory[str]"; expected "AbstractContextManager[Never]"  [arg-type]
+ lib/tests/streamlit/web/cli_test.py:495:29: error: Need type annotation for "https_session"  [var-annotated]
+ lib/tests/streamlit/web/cli_test.py:495:54: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "Session"; expected "AbstractContextManager[Never]"  [arg-type]
+ lib/tests/streamlit/web/cli_test.py:496:20: error: Need type annotation for "proc"  [var-annotated]
+ lib/tests/streamlit/web/cli_test.py:497:17: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "Popen[bytes]"; expected "AbstractContextManager[Never]"  [arg-type]

Jan 13 '24 02:01 github-actions[bot]

Making this a draft. In an ideal world, we would have Self be a supported generic parameter? (Sorry for the noise.)

Jan 13 '24 02:01 NeilGirdhar