mypy icon indicating copy to clipboard operation
mypy copied to clipboard

Fix crash with generic class definition in function

Open Michael0x2a opened this issue 3 years ago • 1 comments

Fixes #12112.

The reason why mypy was crashing with a "Must not defer during final iteration" error in the following snippet:

from typing import TypeVar

def test() -> None:
    T = TypeVar('T', bound='Foo')

    class Foo:
        def bar(self, foo: T) -> None:
            pass

...was because mypy did not seem to be updating the types of the bar callable on each pass: the bind_function_type_variables method in typeanal.py always returned the old type variables instead of using the new updated ones we found by calling self.lookup_qualified(...).

This in turn prevented us from making any forward progress when mypy generated a CallableType containing a placedholder type variable. So, we repeated the semanal passes until we hit the limit and crashed.

I opted to fix this by having the function always return the newly-bound TypeVarLikeType instead. (Hopefully this is safe -- the way mypy likes mutating types always makes it hard to reason about this sort of stuff).

Interestingly, my fix for this bug introduced a regression in one of our existing tests:

from typing import NamedTuple, TypeVar

T = TypeVar("T")
NT = NamedTuple("NT", [("key", int), ("value", T)])

# Test thinks the revealed type should be:
#     def [T] (key: builtins.int, value: T`-1) -> Tuple[builtins.int, T`-1, fallback=__main__.NT[T`-1]]
#
# ...but we started seeing:
#     def [T, _NT <: Tuple[builtins.int, T`-1]] (key: builtins.int, value: T`-1) -> Tuple[builtins.int, T`-1, fallback=test.WTF[T`-1]]
reveal_type(NT)

What seems to be happening here is that during the first pass, we add two type vars to the tvar_scope inside bind_function_type_variables: T with id -1 and _NT with id -2.

But in the second pass, we lose track of the T typevar definition and/or introduce a fresh scope somewhere and infer _NT with id -1 instead?

So now mypy thinks there are two type variables associated with this NamedTuple, which results in the screwed-up type definition.

I wasn't really sure how to fix this, but I thought it was weird that:

  1. We were using negative IDs instead of positive ones. (Class typevars are supposed to use the latter).
  2. We weren't wrapping this whole thing in a new tvar scope frame, given we're nominally synthesizing a new class.

So I did that, and the tests started passing?

I wasn't able to repro this issue for TypedDicts, but opted to introduce a new tvar scope frame there as well for consistency.

Michael0x2a avatar Sep 18 '22 22:09 Michael0x2a

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

github-actions[bot] avatar Sep 18 '22 23:09 github-actions[bot]

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

github-actions[bot] avatar Sep 25 '22 20:09 github-actions[bot]