python
Fix enum attributes are not members
This adds on to the change in https://github.com/python/mypy/pull/17182 and fixes enum attributes being used as members.
@hamdanal / @hauntsaninja I noticed there was the function get_enum_values in mypy types and instead of adding more to the case in try_expanding_sum_type_to_union it seemed like it might make sense to move the changes from https://github.com/python/mypy/pull/17182 there. I don't think the other code touched in that PR can use get_enum_values.
fixes: https://github.com/python/mypy/issues/16730
Diff from mypy_primer, showing the effect of this PR on open source code:
Tanjun (https://github.com/FasterSpeeding/Tanjun)
+ tanjun/_internal/__init__.py:189: error: Statement is unreachable [unreachable]
+ tanjun/annotations.py:2646: error: Statement is unreachable [unreachable]
pytest (https://github.com/pytest-dev/pytest)
+ testing/test_compat.py:242: error: Statement is unreachable [unreachable]
+ testing/test_compat.py:243: error: Unused "type: ignore" comment [unused-ignore]
+ testing/test_compat.py:247: error: Statement is unreachable [unreachable]
flake8 (https://github.com/pycqa/flake8)
+ src/flake8/options/manager.py:170: error: Value of type "str | _ARG" is not indexable [index]
+ src/flake8/options/manager.py:187: error: Argument 1 to "join" of "str" has incompatible type "list[str | _ARG]"; expected "Iterable[str]" [arg-type]
+ src/flake8/options/manager.py:204: error: Incompatible return value type (got "tuple[list[str | _ARG], dict[str, Any]]", expected "tuple[list[str], dict[str, Any]]") [return-value]
aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/client.py:440:28: error: Incompatible types in assignment (expression has type "ClientTimeout | _SENTINEL", variable has type "ClientTimeout") [assignment]
+ aiohttp/client.py:807:55: error: Argument "ws_close" to "ClientWSTimeout" has incompatible type "float | _SENTINEL"; expected "float | None" [arg-type]
+ aiohttp/web_request.py:231:32: error: Argument 1 to "URL" has incompatible type "str | URL | _SENTINEL"; expected "str | URL" [arg-type]
+ aiohttp/web_request.py:236:56: error: Argument 1 to "CIMultiDict" has incompatible type "Mapping[str | istr, str] | CIMultiDict[str] | CIMultiDictProxy[str] | _SENTINEL"; expected "Mapping[str, str] | Mapping[istr, str] | dict[str, str] | dict[istr, str] | MultiMapping[str] | Iterable[tuple[str, str]] | Iterable[tuple[istr, str]]" [arg-type]
+ aiohttp/web_request.py:246:32: error: Incompatible types in assignment (expression has type "str | _SENTINEL", target has type "str") [assignment]
+ aiohttp/web_request.py:248:30: error: Incompatible types in assignment (expression has type "str | _SENTINEL", target has type "str") [assignment]
+ aiohttp/web_request.py:250:32: error: Incompatible types in assignment (expression has type "str | _SENTINEL", target has type "str") [assignment]
+ aiohttp/web_request.py:261:29: error: Argument "client_max_size" to "BaseRequest" has incompatible type "int | _SENTINEL"; expected "int" [arg-type]
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Diff from mypy_primer, showing the effect of this PR on open source code:
Tanjun (https://github.com/FasterSpeeding/Tanjun)
+ tanjun/_internal/__init__.py:189: error: Statement is unreachable [unreachable]
+ tanjun/annotations.py:2646: error: Statement is unreachable [unreachable]
It looks like this is actually related to the ongoing discussion https://github.com/python/typing-council/issues/11
Diff from mypy_primer, showing the effect of this PR on open source code:
Tanjun (https://github.com/FasterSpeeding/Tanjun)
+ tanjun/_internal/__init__.py:189: error: Statement is unreachable [unreachable]
+ tanjun/annotations.py:2646: error: Statement is unreachable [unreachable]
Diff from mypy_primer, showing the effect of this PR on open source code:
Tanjun (https://github.com/FasterSpeeding/Tanjun)
+ tanjun/_internal/__init__.py:189: error: Statement is unreachable [unreachable]
+ tanjun/annotations.py:2646: error: Statement is unreachable [unreachable]
@hamdanal sorry for the delay getting back to this, I went on vacation and then coming back was pretty busy. Work's on break for the week so I figured I could pick this up to keep it from getting delayed any further.
I was having trouble forcing mypy to not use the explicitly provided annotation for an enum member and I will probably have to leave it at this.
I left a comment https://github.com/python/mypy/pull/17207/files#diff-8b48b1eca587106e3806bfa9e14b7f7f344e117b203ffc716ce6fef7f2e68fe6R1610 about the test that was added for https://github.com/python/mypy/issues/11971. The code in question will need to ignore this new error, but as mypy better supports enums the code will need to change as it seems like it's abusing enums a bit.
Diff from mypy_primer, showing the effect of this PR on open source code:
Tanjun (https://github.com/FasterSpeeding/Tanjun)
+ tanjun/_internal/__init__.py:189: error: Statement is unreachable [unreachable]
+ tanjun/annotations.py:2646: error: Statement is unreachable [unreachable]
pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/scope.py:36: error: Type annotations are not allowed for enum members [misc]
+ src/_pytest/scope.py:37: error: Type annotations are not allowed for enum members [misc]
+ src/_pytest/scope.py:38: error: Type annotations are not allowed for enum members [misc]
+ src/_pytest/scope.py:39: error: Type annotations are not allowed for enum members [misc]
+ src/_pytest/scope.py:40: error: Type annotations are not allowed for enum members [misc]
core (https://github.com/home-assistant/core)
+ homeassistant/components/bang_olufsen/const.py:71: error: Type annotations are not allowed for enum members [misc]
+ homeassistant/components/bang_olufsen/const.py:72: error: Type annotations are not allowed for enum members [misc]
+ homeassistant/components/bang_olufsen/const.py:73: error: Type annotations are not allowed for enum members [misc]
+ homeassistant/components/bang_olufsen/const.py:74: error: Type annotations are not allowed for enum members [misc]
+ homeassistant/components/bang_olufsen/const.py:75: error: Type annotations are not allowed for enum members [misc]
+ homeassistant/components/bang_olufsen/const.py:76: error: Type annotations are not allowed for enum members [misc]
+ homeassistant/components/bang_olufsen/const.py:77: error: Type annotations are not allowed for enum members [misc]
+ homeassistant/components/bang_olufsen/const.py:78: error: Type annotations are not allowed for enum members [misc]
+ homeassistant/components/bang_olufsen/const.py:81: error: Type annotations are not allowed for enum members [misc]
+ homeassistant/components/bang_olufsen/const.py:82: error: Type annotations are not allowed for enum members [misc]
+ homeassistant/components/bang_olufsen/const.py:84: error: Type annotations are not allowed for enum members [misc]
jax (https://github.com/google/jax)
+ jaxlib/cpu/_lapack/eig.pyi:20: error: Type annotations are not allowed for enum members [misc]
+ jaxlib/cpu/_lapack/eig.pyi:21: error: Type annotations are not allowed for enum members [misc]
@hamdanal / @JelleZijlstra friendly ping if you have a chance to look over the PR or suggestions on how to fix ^^^
Diff from mypy_primer, showing the effect of this PR on open source code:
Tanjun (https://github.com/FasterSpeeding/Tanjun)
+ tanjun/_internal/__init__.py:189: error: Statement is unreachable [unreachable]
+ tanjun/annotations.py:2646: error: Statement is unreachable [unreachable]
pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/scope.py:36: error: Type annotations are not allowed for enum members [misc]
+ src/_pytest/scope.py:37: error: Type annotations are not allowed for enum members [misc]
+ src/_pytest/scope.py:38: error: Type annotations are not allowed for enum members [misc]
+ src/_pytest/scope.py:39: error: Type annotations are not allowed for enum members [misc]
+ src/_pytest/scope.py:40: error: Type annotations are not allowed for enum members [misc]
jax (https://github.com/google/jax)
+ jaxlib/cpu/_lapack/eig.pyi:20: error: Type annotations are not allowed for enum members [misc]
+ jaxlib/cpu/_lapack/eig.pyi:21: error: Type annotations are not allowed for enum members [misc]
Diff from mypy_primer, showing the effect of this PR on open source code:
Tanjun (https://github.com/FasterSpeeding/Tanjun)
+ tanjun/_internal/__init__.py:189: error: Statement is unreachable [unreachable]
+ tanjun/annotations.py:2646: error: Statement is unreachable [unreachable]
pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/scope.py:36: error: Type annotations are not allowed for enum members [misc]
+ src/_pytest/scope.py:37: error: Type annotations are not allowed for enum members [misc]
+ src/_pytest/scope.py:38: error: Type annotations are not allowed for enum members [misc]
+ src/_pytest/scope.py:39: error: Type annotations are not allowed for enum members [misc]
+ src/_pytest/scope.py:40: error: Type annotations are not allowed for enum members [misc]
jax (https://github.com/google/jax)
+ jaxlib/cpu/_lapack/eig.pyi:20: error: Type annotations are not allowed for enum members [misc]
+ jaxlib/cpu/_lapack/eig.pyi:21: error: Type annotations are not allowed for enum members [misc]
Diff from mypy_primer, showing the effect of this PR on open source code:
Tanjun (https://github.com/FasterSpeeding/Tanjun)
+ tanjun/_internal/__init__.py:189: error: Statement is unreachable [unreachable]
+ tanjun/annotations.py:2646: error: Statement is unreachable [unreachable]
pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/scope.py:36: error: Enum members must be left unannotated [misc]
+ src/_pytest/scope.py:36: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
+ src/_pytest/scope.py:37: error: Enum members must be left unannotated [misc]
+ src/_pytest/scope.py:37: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
+ src/_pytest/scope.py:38: error: Enum members must be left unannotated [misc]
+ src/_pytest/scope.py:38: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
+ src/_pytest/scope.py:39: error: Enum members must be left unannotated [misc]
+ src/_pytest/scope.py:39: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
+ src/_pytest/scope.py:40: error: Enum members must be left unannotated [misc]
+ src/_pytest/scope.py:40: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
jax (https://github.com/google/jax)
+ jaxlib/cpu/_lapack/eig.pyi:20: error: Enum members must be left unannotated [misc]
+ jaxlib/cpu/_lapack/eig.pyi:20: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
+ jaxlib/cpu/_lapack/eig.pyi:21: error: Enum members must be left unannotated [misc]
+ jaxlib/cpu/_lapack/eig.pyi:21: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
I'm confused by the primer hits on Tanjun
I looked a little and couldn't figure out what's going on there either. Both cases involve narrowing an enum attribute inside a loop. I wasn't able to reproduce a similar error with a simple test case, or even by cloning Tanjun and running mypy from this PR directly on those files.
I wasn't able to repro minimised, but I can repro locally via mypy_primer --repo https://github.com/hauntsaninja/mypy --new 46527644f --old 7f3d7f8f1 -k Tanjun. Note that --warn-unreachable is off by default
This is the regression:
λ cat test10.py
import enum
from typing import ClassVar, Literal, cast
class _ParameterKind(enum.IntEnum):
POSITIONAL_ONLY: int
POSITIONAL_OR_KEYWORD: int
VAR_POSITIONAL: int
KEYWORD_ONLY: int
VAR_KEYWORD: int
class Parameter:
POSITIONAL_ONLY: ClassVar[Literal[_ParameterKind.POSITIONAL_ONLY]]
POSITIONAL_OR_KEYWORD: ClassVar[Literal[_ParameterKind.POSITIONAL_OR_KEYWORD]]
VAR_POSITIONAL: ClassVar[Literal[_ParameterKind.VAR_POSITIONAL]]
KEYWORD_ONLY: ClassVar[Literal[_ParameterKind.KEYWORD_ONLY]]
VAR_KEYWORD: ClassVar[Literal[_ParameterKind.VAR_KEYWORD]]
@property
def kind(self) -> _ParameterKind: ...
def get_kwargs() -> None:
parameter = cast(Parameter, ...)
reveal_type(parameter)
reveal_type(parameter.kind)
if parameter.kind is parameter.VAR_KEYWORD:
return
print("should be reachable")
λ mypy test10.py --warn-unreachable --strict --disable-error-code empty-body
test10.py:25: note: Revealed type is "test10.Parameter"
test10.py:26: note: Revealed type is "test10._ParameterKind"
test10.py:31: error: Statement is unreachable [unreachable]
Found 1 error in 1 file (checked 1 source file)
@hauntsaninja I believe this might be related to my comment:
I was having trouble forcing mypy to not use the explicitly provided annotation for an enum member and I will probably have to leave it at this.
But this seems to also not be warning about the:
class _ParameterKind(enum.IntEnum):
POSITIONAL_ONLY: int
POSITIONAL_OR_KEYWORD: int
VAR_POSITIONAL: int
KEYWORD_ONLY: int
VAR_KEYWORD: int
and I can minimally reproduce with:
[case testEnumWithAnnotationOnly]
# flags: --warn-unreachable
import enum
class E(enum.IntEnum):
A: int
B: int
def do_check(value: E) -> None:
reveal_type(value) # N: Revealed type is "__main__.E"
if value is E.A:
return
reveal_type(value) # N: Revealed type is "Literal[__main__.E.B]" # <-- fails
"should be reachable" # <-- fails too
[builtins fixtures/primitives.pyi]
[typing fixtures/typing-full.pyi]
So maybe it's actually that? If I add a A = auto(); B = auto() it does then warn and is not unreachable anymore. I'm adding a test (unfixed) to make sure an enum without values also warns about being annotated.
Diff from mypy_primer, showing the effect of this PR on open source code:
Tanjun (https://github.com/FasterSpeeding/Tanjun)
+ tanjun/_internal/__init__.py:189: error: Statement is unreachable [unreachable]
+ tanjun/annotations.py:2646: error: Statement is unreachable [unreachable]
pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/scope.py:36: error: Enum members must be left unannotated [misc]
+ src/_pytest/scope.py:36: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
+ src/_pytest/scope.py:37: error: Enum members must be left unannotated [misc]
+ src/_pytest/scope.py:37: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
+ src/_pytest/scope.py:38: error: Enum members must be left unannotated [misc]
+ src/_pytest/scope.py:38: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
+ src/_pytest/scope.py:39: error: Enum members must be left unannotated [misc]
+ src/_pytest/scope.py:39: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
+ src/_pytest/scope.py:40: error: Enum members must be left unannotated [misc]
+ src/_pytest/scope.py:40: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
jax (https://github.com/google/jax)
+ jaxlib/cpu/_lapack/eig.pyi:20: error: Enum members must be left unannotated [misc]
+ jaxlib/cpu/_lapack/eig.pyi:20: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
+ jaxlib/cpu/_lapack/eig.pyi:21: error: Enum members must be left unannotated [misc]
+ jaxlib/cpu/_lapack/eig.pyi:21: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
@hauntsaninja I got to the bottom of it, and I fixed it in a way that I figured would have the least amount of knock on effects. According to the spec _ParameterKind should not be warning, but it should be treated as having no members. Which is what I fixed here https://github.com/python/mypy/pull/17207#discussion_r1590072571
The issue though is that try_expanding_sum_type_to_union code would return Never for an enum with no members, and because Never is also effectively the base class of all types Never is _ParameterKind.VAR_KEYWORD, and therefore mypy is "correctly" warning that the code is not reachable. I don't think Never should be treated that way, but I don't want to break other things, so instead I have the try_expanding_sum_type_to_union code check how many members it expands to, and if there are no members it doesn't expand the type and leaves it as is. This may need to stay regardless, but I think the Never is Any may cause other subtle bugs.
This may need to stay regardless
Actually I think if there was a warning if an expression is always true or always false, then this should be removed since the way I implemented things is that it hides Never and that detection would be suppressed here.
Diff from mypy_primer, showing the effect of this PR on open source code:
pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/scope.py:36: error: Enum members must be left unannotated [misc]
+ src/_pytest/scope.py:36: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
+ src/_pytest/scope.py:37: error: Enum members must be left unannotated [misc]
+ src/_pytest/scope.py:37: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
+ src/_pytest/scope.py:38: error: Enum members must be left unannotated [misc]
+ src/_pytest/scope.py:38: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
+ src/_pytest/scope.py:39: error: Enum members must be left unannotated [misc]
+ src/_pytest/scope.py:39: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
+ src/_pytest/scope.py:40: error: Enum members must be left unannotated [misc]
+ src/_pytest/scope.py:40: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
jax (https://github.com/google/jax)
+ jaxlib/cpu/_lapack/eig.pyi:20: error: Enum members must be left unannotated [misc]
+ jaxlib/cpu/_lapack/eig.pyi:20: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
+ jaxlib/cpu/_lapack/eig.pyi:21: error: Enum members must be left unannotated [misc]
+ jaxlib/cpu/_lapack/eig.pyi:21: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members