facebook
Narrowed type is not reflected in nested scopes
from typing import Callable
class Foo:
_window_function: Callable[[str], int] | None
def foo(self) -> None:
if (window_function := self._window_function):
def bar() -> None:
print(window_function('foo'))
bar()
$ pyrefly check g.py
ERROR g.py:9:23-38: Expected a callable, got None [not-callable]
Spotted in Narwhals
Narrowing works with the walrus operator (in this particular case the RHS doesn't work because attribute narrowing doesn't work yet, but the LHS should work)
I think this is a separate issue related to the nested bar function. The narrowed flow type of window_function is not being used inside the body of bar.
Pyrefly behaves correctly when if the window_function call is lifted out of bar
from typing import Callable
class Foo:
_window_function: Callable[[str], int] | None
def foo(self) -> None:
if (window_function := self._window_function):
reveal_type(window_function) # revealed type: (str) -> int [reveal-type]
print(window_function('foo'))
Mypy's behavior here seems to be that if the name comes from the global scope we do not preserve the refinement inside a nested function, but if the name comes from another scope we do preserve the refinement.
def foo() -> int | None:
return None
x = foo()
if x is not None:
reveal_type(x)
def bar() -> None:
reveal_type(x) # mypy thinks this is int | None
I think preserving the refinement isn't safe, but if this is a common pattern maybe we should match mypy here so that we don't give false positives on existing code:
def foo(x: int | None) -> None:
if x is not None:
def bar() -> None:
reveal_type(x) # mypy thinks this is int
bar()
My instinct is that we should not narrow either, at least for now.
Even within the same block where a narrowing is in effect, a write to the same variable would completely overwrite the type (this is why we currently use Anywhere bindings when we look up from a function scope).
In principle there are special cases we could handle, but I think getting them right would be tricky
Actually, I think both mypy and pyright have more subtle behavior here, depending on what happens later in the scope
def foo(x: int | None) -> None:
if x is not None:
def bar() -> None:
reveal_type(x) # mypy thinks this is int
bar()
def foo(x: int | None) -> None:
if x is not None:
def bar() -> None:
reveal_type(x) # mypy thinks this is int | None
x = None
bar()
This also happens if the assignment happens after the call, so I assume it checks the scope after all statements have been processed
def foo(x: int | None) -> None:
if x is not None:
def bar() -> None:
reveal_type(x) # mypy thinks this is int | None
bar()
x = None
The narrowing behavior also differs for the LHS and RHS of a walrus:
def foo(x: int | None) -> None:
if (y := x) is not None:
def bar() -> None:
reveal_type(y) # mypy thinks this is int
reveal_type(x) # mypy thinks this is int | None
I was going to suggest that we keep refinements when we can see that the local is not assigned in the scope where the nested function is defined (if there are other function calls, also discard if their definitions close over the local) — and the examples above suggest that mypy implements this rule or something similar.
For what it’s worth, our paper on Flow describes a slightly more precise variant of the above where the type system tracks writes to closed over variables as an effect, and is able to preserve refined types when it can prove that a function call can not invalidate the refinement based on its tracked effects. https://arxiv.org/abs/1708.08021
a write to the same variable would completely overwrite the type
fwiw in all the example where I've noted this false positive, there are never any writes going on, so I think the mypy / pyright inference looks correct here
Should pyrefly respect the assert type(x) is int kind of type assertions?
pyright does respect such assertions:
Code sample in pyright playground
def foo(x: int | None) -> None:
assert type(x) is int
reveal_type(x) # mypy thinks this is int
BTW is this the right issue to raise up this question or should I better create a separate issue?
Thanks!
P.S.
Just to clarify. Currently the code
def reveal_type(input: int):
pass
def foo(x: int | None) -> None:
assert type(x) is int
reveal_type(x)
triggers the following error in pyrefly:
Argument `int | None` is not assignable to parameter `input` with type `int` in function `reveal_type`
P.P.S.
To further clarify, isinstance assertion works just fine as expected :)
@mexus
should I better create a separate issue?
yes please, that's a type of narrowing that we didn't build yet. it's different for isinstance since it requires the type to be exactly that class and not a subclass.
Another instance:
def f(x: None | str):
assert x is not None
def g() -> str:
return x
https://pyrefly.org/sandbox/?project=N4IgZglgNgpgziAXKOBDAdgEwEYHsAeAdAA4CeSIAOljGAARgAU%2BidAcrujHQD51wAXAE4BKRNTqS6qOHBhCBdfHQhw66XIo5dqEqZlp0A5oxF0AtAD5%2Bw8eikO6QmAICuQ%2B-j2SQAGhCuAtBwJOSIIADEdACqQVAQAqQMrugAxkGccLo09GC4QgC2qAIA%2BuiuBdjyzKwQ6AJmVjZCdg7Obh4MlCBsFVUtdMD4AL7d1H4gZM5gUKSEArgFUBRRAAqk07P8GDgEdKmckEbuxRCchNRRAMow3AAWAgLEcIgA9K9TtLOE%2BUavMOhXphcKk4K8DugjicMoCGPlpAA3VDQVDYWD7Q4QY5CU6cOi4YgwkLUMgCO6ccwI%2BRwM72AC8dG6AGZCABGABMY3QIGG-lQ6QgVIAYtAYBQ0Fg8EQyDygA