mypy
mypy copied to clipboard
python
Unexpected typing failure when passing variable, but no error when passing literal dict
Bug Report
Unexpected typing failure when passing variable, but no error when passing literal dict. Run in strict mode.
To Reproduce
mypy --strict typing_problem.py
typing_problem.py:
import typing
# union includes str
def fn_fails(a: typing.Dict[str, typing.Union[str, float, int]]) -> None:
pass
d = {"abc": 1, "def": 2.0}
fn_fails({"abc": 1, "def": 2.0}) # succeeds
fn_fails(d) # fails
# note: union does NOT include str
def fn_succeeds1(a: typing.Dict[str, typing.Union[float, int]]) -> None:
pass
fn_succeeds1({"abc": 1, "def": 2.0})
fn_succeeds1(d)
# note: union includes str, but type is a Mapping
def fn_succeeds2(a: typing.Mapping[str, typing.Union[str, float, int]]) -> None:
pass
fn_succeeds2({"abc": 1, "def": 2.0})
fn_succeeds2(d)
Expected Behavior
The first two calls to fn_fails seem to be equivalent. I expect both to pass typing checks.
Actual Behavior
The second call to fn_fails produces a typing error.
typing_problem.py:11: error: Argument 1 to "fn_fails" has incompatible type "Dict[str, float]"; expected "Dict[str, Union[str, float, int]]"
typing_problem.py:11: note: "Dict" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance
typing_problem.py:11: note: Consider using "Mapping" instead, which is covariant in the value type
Your Environment
Multiple environments.
% mypy --version
mypy 0.971 (compiled: yes)
I ran into the same issue, turns out its actually not a bug but to be expected.
explained straight to the point in this video: https://www.youtube.com/watch?v=3FTvHnhmd88
mutable collections cannot be covariant, as you can do stuff to the more generic type, that you couldnt do to the more specific version.
in the example in the video: "a bag of apples" is not a subtype of "a bag of fruits", because you can put a banana in the latter, you can't in the former.
in your case:
a: dict[str, int | float] cannot be a subtype of b: dict[str, str | int | float], because if you have the latter, you can do stuff like this: b["asdf"] = "qwer" , which you cannot do in the former.
If dict were immutable, ie. when declared as a: Mapping[str, str | int | float] this mutation is disallowed, and hence can form a subtype.
To go back to the toy example, a "closed bag of apples" is a subtype of a "closed bag of fruits". You can even go as far and say "a bag of apples" is a subtype of a "closed bag of fruits", because you cannot do anything to a "closed bag of fruits" that you couldnt do to a "bag of apples" too.
if you pass the value directly into the function without going over a variable, there is no opportunity for mypy to think that the variable is a dict[str, int | float], but will immediately type it as dict[str, str | int | float] the function argument, hence you dont run into the issue there.
to be honest though that's a bit overkill. python has no notion of immutability like other languages. Mapping is not immutable on the cpython level, it simply does not implement the set method. i usually dont modify my lists and dicts in python anyways, and so do many other python users - for me this feels like nitpicking. Especially if i do type narrowing this can be troublesome, as it will cause unexpected behaviour! (this is pseudocode to highlight the problem, isinstance doesnt support parametrized generics)
if isinstance(data, list[str | int]):
do something
else:
do something else
this would behave like this
["asdf", 5] -> do something
["asdf", "qwer"] -> do something else!
hence i have hidden this behaviour behind a non-default strict flag in my own type-narrowing library https://gitlab.com/dockable/arketip.git
