python
False positive - Type narrowing of class attributes
I'm not super sure if this is a bug or not, but the behavior is different in Mypy and pyright so probably worth discussing it.
To Reproduce
https://mypy-play.net/?mypy=latest&python=3.12&gist=3a34812b38b40ca5cb913ee48daf2c49
from dataclasses import dataclass
from enum import Enum
from typing_extensions import reveal_type
class E(Enum):
ONE: int = 1
TWO: int = 2
@dataclass
class Test:
a: E
def set_a(self, value: E) -> None:
self.a = value
a = Test(E.ONE)
reveal_type(a.a)
assert a.a == E.ONE
reveal_type(a.a)
a.set_a(E.TWO)
reveal_type(a.a)
I ran into this while trying to enable strict_equality on a codebase.
The problem is the last reveal_type call - Mypy claims the type is still Literal[E.ONE], but it's obviously Literal[E.TWO] (or at least E). If strict_equality is enabled, this will cause a type error when asserting in a test, for example.
Pyright doesn't seem to narrow here, claiming the type of a.a is E always.
I don't know what the correct answer is here. Maybe allow narrowing only on local variables? 🤷
- Mypy version used: 1.10.1
- Python version used: 3.8
Yeah, this can be a problem. I'm tagging this as feature, since mypy is working "as designed", though here the behavior is not what is expected.
The narrowing behavior has been around for a very long time, but I think the situation got worse when mypy started narrowing enums on assignment. One option would be to experiment with less eagerly narrowing enums to literal types on attribute assignments/assertions/comparisons. Disabling narrowing of attribute access more widely may cause a lot of new errors, so we need to careful there.
Just to add another example (from aiohttp codebase):
if self._at_eof:
return b""
data = bytearray()
while not self._at_eof:
data.extend(await self.read_chunk(self.chunk_size))
if decode: # unreachable error here
return self.decode(data)
return data
Mypy thinks the code is unreachable because it has narrowed the attribute to False and thinks the loop will never exit.
I don't know how complex it'd be to implement, but I'm think the best approach could be to do the type narrowing, but anytime a method is called on that object, or an await happens to yield to the event loop, then the type narrowing should be reset.
So:
assert foo.bar is True
foo.bar # narrowed to True
foo.something()
foo.bar # bool again
I also ran into this, with a slightly different example:
import sys
class A:
def __init__(self) -> None:
self.had_error: bool = True
def f(a: A) -> None:
a.had_error = False
a = A()
if not a.had_error:
sys.exit(0)
f(a)
reveal_type(a.had_error)
However, I did notice that pyright makes exactly the same mistake.
information: Type of "a.had_error" is "Literal[True]"
I guess the type narrowing section should be expanded.
Expecting a type checker to know that an arbitrary function modified an attribute seems unreasonable to me. I think this should be limited to methods resetting the narrowing (otherwise the narrowing would be reset for everything, such as a print() call, at which point it become a lot less useful as a feature).
@Dreamsorcerer Honestly, I completely missed the existing responses here, because GH collapsed them.
To have "sound" type narrowing, mypy needs to know which functions are mutating arguments. This information is not available. Your suggestion seems to be that we approximate this information:
- if a function is a method, then a function is mutating,
- otherwise a function is read-only.
In my opinion, this does more harm than good. Everyone would need to know this new aspect of mypy. Moreover, it introduces other errors (for example read-only methods then be misclassified; and functions, which call mutating methods, would still be misclassified).
I still think it would be good if the incorrect type narrowing was at least opt-out. But if this is not wanted by the maintainers, then we should simply document this edge case, and move on.
Note that this isn't enum-specific.
class Test:
a: int | None = 0
def set_a(self, value: int | None) -> None:
self.a = value
a = Test()
reveal_type(a.a) # N: Revealed type is "builtins.int | None"
assert a.a is not None
reveal_type(a.a) # N: Revealed type is "builtins.int"
a.set_a(None)
# Oops
reveal_type(a.a) # N: Revealed type is "builtins.int"
https://mypy-play.net/?mypy=latest&python=3.12&gist=9d9394ac88293d89cb56b921b85443c6&flags=strict%2Cwarn-unreachable
This issue is ultimately about mypy inability to invalidate attribute type narrowing when a mutating method is called (or some other potentially mutating event happens, as in #19608).
Similar issues: https://github.com/python/mypy/issues/14543 https://github.com/python/mypy/issues/14010
