python
Type refinement does not work with `None in [a, b, c]`, `any()`, or `all()` checks
When checking if multiple variables are not None using the syntax None in [a, b, c], mypy fails to refine the types of these variables to non-Optional types after this check. This issue persists even when using any() or all() for similar checks, preventing mypy from correctly inferring that the variables cannot be None past these checks, despite the code logically ensuring that these variables are not None.
Code to Reproduce:
from typing import Optional
def init_vars(
a: Optional[str] = None,
b: Optional[int] = None,
c: Optional[int] = None,
) -> None:
if None in [a, b, c]:
raise ValueError("a, b, and c cannot be None")
# Expected: a, b, c should not be Optional types beyond this point
# Actual: mypy still treats a, b, c as Optional types
print(a.upper()) # mypy error: Item "None" of "Optional[str]" has no attribute "upper"
print(b + 1) # mypy error: Unsupported operand type(s) for +: 'int' and 'NoneType'
print(c + 1) # mypy error: Unsupported operand type(s) for +: 'int' and 'NoneType'
And here is another example with the all function (with any it is quite similar):
def init_vars_with_all(
a: Optional[str] = None,
b: Optional[int] = None,
c: Optional[int] = None,
) -> None:
if not all(x is not None for x in [a, b, c]):
raise ValueError("a, b, and c cannot be None")
# Expected: a, b, c should not be Optional types beyond this point
# Actual: mypy still treats a, b, c as Optional types
print(a.upper()) # mypy error: same as above
print(b + 1) # mypy error: same as above
print(c + 1) # mypy error: same as above
Expected Behavior:
After the checks if None in [a, b, c]: or using all(), mypy should refine the types of a, b, and c to str, int, and int respectively, acknowledging that they cannot be None.
Actual Behavior:
mypy does not refine the types and continues to treat a, b, and c as Optional[str], Optional[int], and Optional[int] respectively. This results in type errors when attempting to use these variables in a context that does not allow None.
Additional Information:
mypy version: 1.9.0 compiled Python version: 3.10 No flags or configurations are altering the default behavior of mypy in this instance.
This behavior suggests a limitation in mypy's type inference system when it comes to using collective None checks with list membership or aggregate functions like any() and all(). An enhancement to allow type refinement in such cases would significantly improve usability in scenarios involving initialization checks for multiple variables.
Each type narrowing pattern supported by a type checker requires specialized logic. Mypy doesn't currently support the two patterns shown above in your code sample: None in [a, b, c] and not all(x is not None for x in [a, b, c]. The second one is quite complex and unusual. The first is one I've seen before, although it's still not very common.
Mypy does support a is None or b is None or c is None, which is admittedly a bit more verbose than the first check in your code.
In any case, this should probably be considered a feature request, not a bug report.
It would be nice if the scope could be expanded to include other cases:
- Enums: https://github.com/python/mypy/issues/17152?
- Primitives:
1 in (a, b, c) - Literals/Finals:
_MP: Final = "__"; _MP in (a, b, c)
Pylint offers https://pylint.readthedocs.io/en/latest/user_guide/messages/refactor/consider-using-in.html as a suggestion
