python
Callables like `operator.call` produce `arg-type` error with generic functions
Bug Report
operator.call and similarly-typed callables produce an arg-type error when used with many generic functions.
Additionally, if the arg-type error is # type: ignore'd, the inferred type is a TypeVar (removed from its scope!) rather than Any.
To Reproduce
import operator
from collections.abc import Callable
from typing import TypeVar
from typing import assert_type
from typing import reveal_type
T = TypeVar("T")
T0 = TypeVar("T0")
U = TypeVar("U") # this is redundant, but it helps understand mypy's output
def call2(cb: Callable[[T0], T], arg0: T0, /) -> T:
# similar to operator.call, but only takes one posarg (doesn't use ParamSpec)
return cb(arg0)
def ident(x: U) -> U:
return x
assert_type(operator.call(ident, "foo"), str)
# error: Expression is of type "U", not "str" [assert-type]
# error: Argument 2 to "call" has incompatible type "str"; expected "U" [arg-type]
assert_type(call2(ident, "foo"), str)
# error: Expression is of type "U", not "str" [assert-type]
# error: Argument 1 to "call" has incompatible type "Callable[[U], U]"; expected "Callable[[str], U]" [arg-type]
x = operator.call(ident, "foo") # type: ignore[arg-type]
reveal_type(x) # should be inferred as Any
# note: Revealed type is "U`-1"
(https://mypy-play.net/?mypy=latest&python=3.11&gist=a66524b9632714157bd87ef422725e41)
Expected Behavior
The inferred type of operator.call(ident, "foo") should match the inferred type of ident("foo"). (Aside: should this be str or Literal["foo"]?)
Comparing to other type checkers, call2(ident, "foo") works with pyre (inferred as Literal["foo"]), sort-of-works with pyright (inferred as U@ident | str), and doesn't work with pytype (inferred as Any).
Also, if # type: ignore[arg-type] is used on the arg-type error, the inferred type should by Any, not a TypeVar removed from its scope.
Actual Behavior
See comments in the reproducer above.
Your Environment
- Mypy version used: 0.991 (compiled) and latest master (uncompiled)
- Mypy command-line flags: none required to reproduce
- Mypy configuration options from
mypy.ini(and other config files): none required to reproduce - Python version used: 3.11, 3.10, 3.9, 3.8, 3.7
It works when the definition of call2 only takes a single TypeVar:
T = TypeVar("T")
U = TypeVar("U")
def call2(cb: Callable[[T], T], arg0: T, /) -> T:
return cb(arg0)
def ident(x: U) -> U:
return x
assert_type(call2(ident, "foo"), str)
So, I guess mypy has trouble realizing it should assign U to both T0 and T in your example?
yeah, the problem seems to be that Mypy isn't binding TypeVars of generic functions when they are callback functions (rather than being the function being called).
overloads aren't getting resolved either:
from operator import call
from typing import Literal
from typing import overload
@overload
def foo(*, flag: Literal[True], extra_if_flag: int = ...) -> None:
...
@overload
def foo(*, flag: Literal[False] = ...) -> None:
...
def foo(*, flag: bool = False, extra_if_flag: int = 0) -> None:
...
foo()
call(foo) # error: Missing named argument "flag" for "call" [call-arg]
foo(flag=False)
call(foo, flag=False) # error: Argument "flag" to "call" has incompatible type "Literal[False]"; expected "Literal[True]" [arg-type]
foo(flag=True)
call(foo, flag=True)
foo(flag=True, extra_if_flag=0)
call(foo, flag=True, extra_if_flag=0)
(https://mypy-play.net/?mypy=1.1.1&python=3.11&gist=590d1fbbfdc8afd03a82d7877bbea61b)
The original example (with generics) passes on current master with --new-type-inference. The other example from comments (with overloads) still fails some calls, with errors like:
test.py:24: error: Argument 1 to "call" has incompatible type overloaded function; expected "Callable[[bool], None]" [arg-type]
test.py:27: error: Argument 1 to "call" has incompatible type overloaded function; expected "Callable[[bool], None]" [arg-type]
test.py:30: error: Argument 1 to "call" has incompatible type overloaded function; expected "Callable[[bool, int], None]" [arg-type]
One can argue that mypy is overly strict here (the error goes away if you use regular position-or-name arguments in definition of foo()). But handling argument kinds is tricky with ParamSpec (unless we admit some loss of type safety in similar but incorrect examples).
I recommend opening a specific separate issue about this remaining edge case if you think it is important. (cc @JukkaL this is precisely the corner case I mentioned in #15896)
