mypy
mypy copied to clipboard
Type Narrowing failure in 0.981
I'm seeing an Incompatible return value type error in 0.981 that wasn't present in previous versions of mypy.
To Reproduce
from typing import TypeVar
T = TypeVar("T", dict, float)
def method(x: T) -> T:
if isinstance(x, dict):
return {}
else:
return 0.0
Expected Behavior
% mypy --version
mypy 0.971 (compiled: yes)
% mypy test-mypy-error.py
Success: no issues found in 1 source file
Actual Behavior
% mypy --version
mypy 0.981 (compiled: yes)
% mypy test-mypy-error.py
[Success: no issues found in 1 source file](test-mypy-error.py:7: error: Incompatible return value type (got "Dict[<nothing>, <nothing>]", expected "float"))
Your Environment mypy version 0.981 python version 3.9.13
Interesting, mypy_primer -p ~/dev/mypy_primer/test.py --bisect --new v0.981 --old v0.971 --debug
bisects this to https://github.com/python/mypy/pull/13386
Looks like it's from https://github.com/python/typeshed/pull/8465, cc @sobolevn
Very strange 😨
How can __hash__
affect this?
Yeah, it's pretty strange, but it's true. Fixes itself on master if you add __hash__
back to float. These dunders...
It does not feel right. It looks like it exposes some other bug. I will have a look.
So, I am diving into it.
First of all, I was able to reproduce this problem. Secondly, this works:
from typing import TypeVar
T = TypeVar("T", dict, int)
def method(x: T) -> T:
if isinstance(x, dict):
return {}
else:
return 0
Also, set
and list
work.
But, float
and str
does not work.
At this point I am sure that this is a bug.
Revealed types
Let's reveal types:
from typing import TypeVar
T = TypeVar("T", dict, str)
def method(x: T) -> T:
if isinstance(x, dict):
reveal_type(x)
return {}
else:
reveal_type(x)
return 'a'
Output:
ex.py:7: note: Revealed type is "builtins.dict[Any, Any]"
ex.py:7: note: Revealed type is "ex.<subclass of "dict" and "str">"
ex.py:8: error: Incompatible return value type (got "Dict[<nothing>, <nothing>]", expected "str") [return-value]
ex.py:10: note: Revealed type is "builtins.str"
Looks like Revealed type is "ex.<subclass of "dict" and "str">"
should not ever happen.
What about types that do work?
from typing import TypeVar
T = TypeVar("T", dict, int)
def method(x: T) -> T:
if isinstance(x, dict):
reveal_type(x)
return {}
else:
reveal_type(x)
return 1
Outputs:
» mypy ex.py --show-traceback
ex.py:7: note: Revealed type is "builtins.dict[Any, Any]"
ex.py:10: note: Revealed type is "builtins.int"
Success: no issues found in 1 source file
So, let's find why the intersection of two instances is created in the first place 🤔
.intersect_instances
This happens, because self.intersect_instances((v, t), ctx)
works this way:
Intersecting: builtins.int builtins.dict[Any, Any]
Result: None
Intersecting: builtins.str builtins.dict[Any, Any]
Result: ex.<subclass of "dict" and "str">
Possibly related: https://github.com/python/mypy/issues/13956 Even though this fails also for previous mypy versions.
I am not sure if this is related but I wanted to add another data point on 0.991
@classmethod
def from_df(cls, df: pd.DataFrame | pd.Series) -> "Sites":
if isinstance(df, pd.Series):
return cls.from_sitemap(cast(SiteDict, df.to_dict()))
assert isinstance(df, pd.DataFrame) # for mypy type failure
df.rename({df.columns[0]: "name"}, axis=1, inplace=True)
Error:
error: No overload variant of "rename" of "Series" matches argument types "Dict[Union[str, bytes, date, timedelta, int, float, complex, Any], str]", "int", "bool" [call-overload]
df.rename({df.columns[0]: "name"}, axis=1, inplace=True)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Without the assert isinstance(df, pd.DataFrame) # for mypy type failure
Mypy falsely warns that Series.rename
does not have the proper overload, but df
should already be narrowed to a pd.DataFrame
by that point. Pylance/pyright gets it right.
Environment:
mypy 0.991 (compiled: yes)
Python 3.10.6
Options:
--allow-redefinition --allow-untyped-globals --ignore-missing-imports --implicit-reexport --enable-incomplete-feature=Unpack
Are we sure the initial report reveals a bug or is this behaviour by design? If I am right, we can consider it only a bug if the order of the type variable constraints matters. Then Mypy should behave as follows:
from typing import TypeVar
class A: ...
class B: v = 1
class C(A, B): ...
T1 = TypeVar("T1", A, B)
def f1(x: T1) -> T1:
if isinstance(x, A):
return A() # no error
return B()
T2 = TypeVar("T2", B, A)
def f2(x: T2) -> T2:
if isinstance(x, A):
return A() # error: Incompatible return value type (got "A", expected "B") [return-value]
return B()
f1(C()).v # error: "A" has no attribute "v" [attr-defined]
f2(C()).v
Are there any promises that Mypy or other type checkers prioritise the first over the second type variable constraint when estimating return types in the context of multiple inheritance?
If we agree that order matters (and if I do not miss other potential problems here), adjusting Mypy should be manageable with reasonable effort.
It seems to be a known issue. I found the following test case (added by @Michael0x2a):
[case testIsInstanceAdHocIntersectionGenericsWithValuesDirectReturn]
# flags: --warn-unreachable
from typing import TypeVar
class A:
attr: int
class B:
attr: int
class C:
attr: str
T1 = TypeVar('T1', A, B)
def f1(x: T1) -> T1:
if isinstance(x, A):
# The error message is confusing, but we indeed do run into problems if
# 'x' is a subclass of A and B
return A() # E: Incompatible return value type (got "A", expected "B")
else:
return B()
T2 = TypeVar('T2', B, C)
def f2(x: T2) -> T2:
if isinstance(x, B):
# In contrast, it's impossible for a subclass of "B" and "C" to
# exist, so this is fine
return B()
else:
return C()
[builtins fixtures/isinstance.pyi]