mypy icon indicating copy to clipboard operation
mypy copied to clipboard

Fix `TypeIs` negative narrowing of union of generics

Open brianschubert opened this issue 1 year ago • 4 comments

Fixes #18009

Modelling the runtime behavior of isinstance (which erases generic type arguments) isn't applicable to TypeIs. This PR adds a flag so that we can skip that logic deep inside conditional_types_with_intersection.

It's a little awkward having to pass a flag down through so many call levels, but I can't think of a better way.

brianschubert avatar Nov 26 '24 19:11 brianschubert

Diff from mypy_primer, showing the effect of this PR on open source code:

prefect (https://github.com/PrefectHQ/prefect)
+ src/prefect/tasks.py:825: error: Incompatible return value type (got "Coroutine[Any, Any, TaskRun]", expected "TaskRun")  [return-value]
+ src/prefect/tasks.py:825: note: Maybe you forgot to use "await"?

pydantic (https://github.com/pydantic/pydantic)
+ pydantic/fields.py:606: error: Redundant cast to "Callable[[], Any]"  [redundant-cast]

jinja (https://github.com/pallets/jinja)
+ src/jinja2/async_utils.py:70: error: Incompatible return value type (got "Union[Awaitable[V], V]", expected "V")  [return-value]

pytest (https://github.com/pytest-dev/pytest)
+ testing/test_monkeypatch.py:418: error: Unused "type: ignore" comment  [unused-ignore]

discord.py (https://github.com/Rapptz/discord.py)
- discord/utils.py:716: error: Unused "type: ignore" comment  [unused-ignore]

steam.py (https://github.com/Gobot1234/steam.py)
+ steam/utils.py:868: error: Incompatible return value type (got "Any | _T | Awaitable[_T]", expected "_T")  [return-value]

core (https://github.com/home-assistant/core)
+ homeassistant/core.py:673: error: Argument 1 to "HassJob" has incompatible type "Callable[[VarArg(*_Ts)], Coroutine[Any, Any, _R] | _R] | Coroutine[Any, Any, _R]"; expected "Callable[[VarArg(*_Ts)], Coroutine[Any, Any, _R] | _R]"  [arg-type]
+ homeassistant/core.py:673: error: Argument 1 to "HassJob" has incompatible type "Callable[[VarArg(*_Ts)], Coroutine[Any, Any, _R] | _R] | Coroutine[Any, Any, _R]"; expected "Callable[[VarArg(Any), KwArg(Any)], Coroutine[Any, Any, _R]]"  [arg-type]
+ homeassistant/core.py:1000: error: Argument 1 to "HassJob" has incompatible type "Callable[[VarArg(*_Ts)], Coroutine[Any, Any, _R] | _R] | Coroutine[Any, Any, _R]"; expected "Callable[[VarArg(*_Ts)], Coroutine[Any, Any, _R] | _R]"  [arg-type]
+ homeassistant/core.py:1000: error: Argument 1 to "HassJob" has incompatible type "Callable[[VarArg(*_Ts)], Coroutine[Any, Any, _R] | _R] | Coroutine[Any, Any, _R]"; expected "Callable[[VarArg(Any), KwArg(Any)], Coroutine[Any, Any, _R]]"  [arg-type]

github-actions[bot] avatar Nov 26 '24 20:11 github-actions[bot]

Diff from mypy_primer, showing the effect of this PR on open source code:

pydantic (https://github.com/pydantic/pydantic)
+ pydantic/fields.py:606: error: Redundant cast to "Callable[[], Any]"  [redundant-cast]

pytest (https://github.com/pytest-dev/pytest)
+ testing/test_monkeypatch.py:418: error: Unused "type: ignore" comment  [unused-ignore]

github-actions[bot] avatar Nov 26 '24 21:11 github-actions[bot]

Diff from mypy_primer, showing the effect of this PR on open source code:

pytest (https://github.com/pytest-dev/pytest)
+ testing/test_monkeypatch.py:420: error: Unused "type: ignore" comment  [unused-ignore]

github-actions[bot] avatar Jun 12 '25 13:06 github-actions[bot]

More issues might be closed by this PR: https://github.com/sterliakov/mypy-issues/issues/70 I didn't look deeper, but all four look related enough

sterliakov avatar Jun 12 '25 19:06 sterliakov

Diff from mypy_primer, showing the effect of this PR on open source code:

pytest (https://github.com/pytest-dev/pytest)
+ testing/test_monkeypatch.py:420: error: Unused "type: ignore" comment  [unused-ignore]

starlette (https://github.com/encode/starlette)
+ starlette/middleware/errors.py:178: error: Argument 1 to "run_in_threadpool" has incompatible type "Callable[[Request, Exception], Response | Awaitable[Response]] | Callable[[WebSocket, Exception], Awaitable[None]]"; expected "Callable[[Request, Exception], Response]"  [arg-type]
+ starlette/_exception_handler.py:61: error: Argument 1 to "run_in_threadpool" has incompatible type "Callable[[Request, Exception], Response | Awaitable[Response]] | Callable[[WebSocket, Exception], Awaitable[None]]"; expected "Callable[[Request | WebSocket, Exception], Any]"  [arg-type]
+ starlette/routing.py:68: error: Incompatible types in assignment (expression has type "Callable[..., Awaitable[Any]] | partial[Coroutine[Any, Any, Awaitable[Response] | Response]]", variable has type "Callable[[Request], Awaitable[Response]]")  [assignment]
+ starlette/routing.py:68: error: Too few arguments for "run_in_threadpool"  [call-arg]

github-actions[bot] avatar Jun 20 '25 18:06 github-actions[bot]

Diff from mypy_primer, showing the effect of this PR on open source code:

pytest (https://github.com/pytest-dev/pytest)
+ testing/test_monkeypatch.py:420: error: Unused "type: ignore" comment  [unused-ignore]

starlette (https://github.com/encode/starlette)
+ starlette/middleware/errors.py:178: error: Argument 1 to "run_in_threadpool" has incompatible type "Callable[[Request, Exception], Response | Awaitable[Response]] | Callable[[WebSocket, Exception], Awaitable[None]]"; expected "Callable[[Request, Exception], Response]"  [arg-type]
+ starlette/_exception_handler.py:61: error: Argument 1 to "run_in_threadpool" has incompatible type "Callable[[Request, Exception], Response | Awaitable[Response]] | Callable[[WebSocket, Exception], Awaitable[None]]"; expected "Callable[[Request | WebSocket, Exception], Any]"  [arg-type]
+ starlette/routing.py:68: error: Incompatible types in assignment (expression has type "Callable[..., Awaitable[Any]] | partial[Coroutine[Any, Any, Awaitable[Response] | Response]]", variable has type "Callable[[Request], Awaitable[Response]]")  [assignment]
+ starlette/routing.py:68: error: Too few arguments for "run_in_threadpool"  [call-arg]

github-actions[bot] avatar Jun 20 '25 20:06 github-actions[bot]

Just confirming (a little late), the starlette hit looks correct to me too.

I think it boils down to something like this:

from collections.abc import Callable
from typing import Any
from typing_extensions import TypeIs

class Foo[T]: pass

def is_foo_factory(x: object) -> TypeIs[Callable[..., Foo[Any]]]: ...

# ===== Before  =====

def f(x: Callable[..., int | Foo[int]] | Callable[..., Foo[str]]) -> None:
    if is_foo_factory(x):
        reveal_type(x)  # N: Revealed type is "def (*Any, **Any) -> SCRATCH.Foo[Any]"
    else:
        reveal_type(x)  # E: Statement is unreachable 


# ===== After  =====

def f(x: Callable[..., int | Foo[int]] | Callable[..., Foo[str]]) -> None:
    if is_foo_factory(x):
        reveal_type(x)  # N: Revealed type is "def (*Any, **Any) -> SCRATCH.Foo[Any]"
    else:
        reveal_type(x)  # N: Revealed type is "def (*Any, **Any) -> builtins.int | SCRATCH.Foo[builtins.int]"

which all looks right to me. From what I can tell, the logic in starlette is expecting the else branch to also narrow Callable[..., int | Foo[int]] to Callable[..., int], which wouldn't be right.

brianschubert avatar Jun 21 '25 20:06 brianschubert