python
Problem with @overload and Iterator/Generator
See last line (Playground):
from collections.abc import Iterator, Generator
from typing import Any, Iterable, TypeVar, overload
from typing_extensions import assert_type
T = TypeVar("T")
IterableT = TypeVar("IterableT", bound=Iterable)
@overload
def read_iterator(iterable: Iterator[T]) -> list[T]: ...
@overload
def read_iterator(iterable: IterableT) -> IterableT: ...
def read_iterator(iterable: Iterable) -> Iterable:
if isinstance(iterable, Iterator):
return list(iterable)
else:
return iterable
def _create_generator() -> Generator[int, Any, None]:
for i in range(3):
yield i
def test_read_iterator() -> None:
an_iterator = iter(range(3))
assert_type(read_iterator(an_iterator), list[int])
a_range, a_list, a_tuple, a_set, a_dict = range(3), [1, 2, 3], (1, 2, 3), {1, 2, 3}, {"a": 1}
assert_type(read_iterator(a_range), range)
assert_type(read_iterator(a_list), list[int])
assert_type(read_iterator(a_tuple), tuple[int, int, int])
assert_type(read_iterator(a_set), set[int])
assert_type(read_iterator(a_dict), dict[str, int])
a_generator = _create_generator()
assert_type(a_generator, Generator[int, Any, None])
assert_type(read_iterator(a_generator), list[int]) # error: Expression is of type "Any", not "list[int]" [assert-type]
Mypy doesn't like this code:
error: Expression is of type "Any", not "list[int]" [assert-type]
Please remember that Generator is a subtype of Iterator:
>>> isinstance(a_generator, Iterator)
True
I think this is a bug in Mypy. The Generator should match the signature of def read_iterator(iterable: Iterator[T]) -> list[T]
Versions:
> python -V
Python 3.11.1
> mypy --version
mypy 1.11.2 (compiled: yes)
❯ system_profiler SPSoftwareDataType
Software:
System Software Overview:
System Version: macOS 13.6.7 (22G720)
Kernel Version: Darwin 22.6.0
Related: https://github.com/python/typing/issues/253
In VSCode the returned type of read_iterator(a_generator) is list[int] with not problem:
This looks related to the case mypy has where if multiple overloads match due to the presence of Any, in some situations it will infer Any. It looks like that code path is getting triggered by the Any in Generator's send type. (I think the semantics here are a little ad hoc, the Typing Council does have specifying behaviour here on its list)
