Fix checking multiple assignments based on tuple unpacking involving partially initialised variables (Fixes #12915).

Fix checking multiple assignments based on tuple unpacking involving partially initialised variables (Fixes #12915).

This proposal is an alternative to #14423. Similar to #14423, the main idea is to convert unions of tuples to tuples of (simplified) unions during multi-assignment checks. In addition, it extends this idea to other iterable types, which allows removing the undefined_rvalue logic and the no_partial_types logic. Hence, the problem reported in #12915 with partially initialised variables should be fixed for unions that combine, for example, tuples and lists, as well.

Besides the new test case also provided by #14423 (testDefinePartiallyInitialisedVariableDuringTupleUnpacking), this commit also adds the test cases testUnionUnpackingIncludingListPackingSameItemTypes, testUnionUnpackingIncludingListPackingDifferentItemTypes, and testUnionUnpackingIncludingListPackingForVariousItemTypes.

Diff from mypy_primer, showing the effect of this PR on open source code:

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/test.py:1274: error: Cannot determine type of "headers"  [has-type]
+ src/werkzeug/routing/map.py:735: error: Cannot determine type of "path"  [has-type]
+ src/werkzeug/routing/map.py:735: error: Cannot determine type of "domain_part"  [has-type]

python-chess (https://github.com/niklasf/python-chess)
+ chess/pgn.py:481: error: Cannot determine type of "tail"  [has-type]
+ chess/pgn.py:481: error: Cannot determine type of "head"  [has-type]

pylint (https://github.com/pycqa/pylint)
+ pylint/checkers/base_checker.py:228: error: Cannot determine type of "msg"  [has-type]
+ pylint/checkers/base_checker.py:228: error: Cannot determine type of "descr"  [has-type]
+ pylint/checkers/base_checker.py:228: error: Cannot determine type of "symbol"  [has-type]

steam.py (https://github.com/Gobot1234/steam.py)
- steam/game_server.py:285: error: "None" not callable  [misc]
+ steam/game_server.py:283: error: Cannot determine type of "op"  [has-type]
+ steam/game_server.py:284: error: Cannot determine type of "query_1"  [has-type]
+ steam/game_server.py:285: error: Cannot determine type of "query_2"  [has-type]
+ steam/game_server.py:285: error: Cannot determine type of "query_1"  [has-type]

aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/client_reqrep.py:348: error: Cannot determine type of "key"  [has-type]
+ aiohttp/client_reqrep.py:349: error: Cannot determine type of "key"  [has-type]
+ aiohttp/client_reqrep.py:349: error: Cannot determine type of "value"  [has-type]
+ aiohttp/client_reqrep.py:351: error: Cannot determine type of "key"  [has-type]
+ aiohttp/client_reqrep.py:351: error: Cannot determine type of "value"  [has-type]

bokeh (https://github.com/bokeh/bokeh)
+ src/bokeh/colors/color.py: note: In member "from_tuple" of class "RGB":
+ src/bokeh/colors/color.py:302:24: error: Cannot determine type of "r"  [has-type]
+ src/bokeh/colors/color.py:302:27: error: Cannot determine type of "g"  [has-type]
+ src/bokeh/colors/color.py:302:30: error: Cannot determine type of "b"  [has-type]
+ src/bokeh/colors/color.py:305:24: error: Cannot determine type of "r"  [has-type]
+ src/bokeh/colors/color.py:305:27: error: Cannot determine type of "g"  [has-type]
+ src/bokeh/colors/color.py:305:30: error: Cannot determine type of "b"  [has-type]
+ src/bokeh/colors/color.py:305:33: error: Cannot determine type of "a"  [has-type]

pydantic (https://github.com/samuelcolvin/pydantic)
+ pydantic/color.py:159: error: Cannot determine type of "s"  [has-type]
+ pydantic/color.py:159: error: Cannot determine type of "li"  [has-type]
+ pydantic/color.py:159: error: Cannot determine type of "h"  [has-type]
+ pydantic/color.py:162: error: Cannot determine type of "s"  [has-type]
+ pydantic/color.py:162: error: Cannot determine type of "li"  [has-type]
+ pydantic/color.py:162: error: Cannot determine type of "h"  [has-type]
+ pydantic/color.py:162: error: Cannot determine type of "a"  [has-type]

I checked the primer diff for pylint and simplified the relevant code. The error happens in the last line:

MessageDefinitionTuple = Union[
    Tuple[str, str, str],
    Tuple[str, str, str, ExtraMessageOptions],
]

options: ExtraMessageOptions = {}
if len(msg_tuple) == 4:
    (msg, symbol, descr, options) = msg_tuple  # type: ignore[misc]
elif len(msg_tuple) == 3:
    (msg, symbol, descr) = msg_tuple  # type: ignore[misc]
else:
    raise InvalidMessageError("invalid error message")
return MessageDefinition(self, msgid, msg, descr, symbol, **options)

So, the new error message seems valid.

The differences reported for steam are also related to an ignored tuple error:

_raw: tuple[Query[Any], Operator, T_co] | tuple[str]

self._raw = raw  # type: ignore  # can't tell if this is a pyright bug

query_1, op, query_2 = self._raw

return op.format(
    query_1.query,
    query_2.query if isinstance(query_2, Query) else query_1._callback(query_2),
)

However, this reminds me that I forgot to consider ellipsis cases, e.g. Tuple[int, ...].

However, this reminds me that I forgot to consider ellipsis cases, e.g. Tuple[int, ...].

It's okay. Mypy seems to handle Tuple[int, ...] as Instance, so this case is already covered by isinstance(item, Instance) and self.type_is_iterable(item) for handling iterables like List[int]. Only fixed-size tuples seem to be translated to TupleType. I will adjust a test case nevertheless.

Last example pydantic:

HslColorTuple = Union[Tuple[float, float, float], Tuple[float, float, float, float]]

def as_hsl_tuple(self, *, alpha: Optional[bool] = None) -> HslColorTuple:
    ...

def as_hsl(self) -> str:
    if self._rgba.alpha is None:
        h, s, li = self.as_hsl_tuple(alpha=False)  # type: ignore
        return f'hsl({h * 360:0.0f}, {s:0.0%}, {li:0.0%})'
    else:
        h, s, li, a = self.as_hsl_tuple(alpha=True)  # type: ignore
        return f'hsl({h * 360:0.0f}, {s:0.0%}, {li:0.0%}, {round(a, 2)})'

Still the same reason.

Diff from mypy_primer, showing the effect of this PR on open source code:

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/test.py:1274: error: Cannot determine type of "headers"  [has-type]
+ src/werkzeug/routing/map.py:735: error: Cannot determine type of "path"  [has-type]
+ src/werkzeug/routing/map.py:735: error: Cannot determine type of "domain_part"  [has-type]

python-chess (https://github.com/niklasf/python-chess)
+ chess/pgn.py:481: error: Cannot determine type of "tail"  [has-type]
+ chess/pgn.py:481: error: Cannot determine type of "head"  [has-type]

aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/client_reqrep.py:348: error: Cannot determine type of "key"  [has-type]
+ aiohttp/client_reqrep.py:349: error: Cannot determine type of "key"  [has-type]
+ aiohttp/client_reqrep.py:349: error: Cannot determine type of "value"  [has-type]
+ aiohttp/client_reqrep.py:351: error: Cannot determine type of "key"  [has-type]
+ aiohttp/client_reqrep.py:351: error: Cannot determine type of "value"  [has-type]

steam.py (https://github.com/Gobot1234/steam.py)
- steam/game_server.py:285: error: "None" not callable  [misc]
+ steam/game_server.py:283: error: Cannot determine type of "op"  [has-type]
+ steam/game_server.py:284: error: Cannot determine type of "query_1"  [has-type]
+ steam/game_server.py:285: error: Cannot determine type of "query_2"  [has-type]
+ steam/game_server.py:285: error: Cannot determine type of "query_1"  [has-type]

pylint (https://github.com/pycqa/pylint)
+ pylint/checkers/base_checker.py:228: error: Cannot determine type of "msg"  [has-type]
+ pylint/checkers/base_checker.py:228: error: Cannot determine type of "descr"  [has-type]
+ pylint/checkers/base_checker.py:228: error: Cannot determine type of "symbol"  [has-type]

bokeh (https://github.com/bokeh/bokeh)
+ src/bokeh/colors/color.py: note: In member "from_tuple" of class "RGB":
+ src/bokeh/colors/color.py:302:24: error: Cannot determine type of "r"  [has-type]
+ src/bokeh/colors/color.py:302:27: error: Cannot determine type of "g"  [has-type]
+ src/bokeh/colors/color.py:302:30: error: Cannot determine type of "b"  [has-type]
+ src/bokeh/colors/color.py:305:24: error: Cannot determine type of "r"  [has-type]
+ src/bokeh/colors/color.py:305:27: error: Cannot determine type of "g"  [has-type]
+ src/bokeh/colors/color.py:305:30: error: Cannot determine type of "b"  [has-type]
+ src/bokeh/colors/color.py:305:33: error: Cannot determine type of "a"  [has-type]

pydantic (https://github.com/samuelcolvin/pydantic)
+ pydantic/color.py:159: error: Cannot determine type of "s"  [has-type]
+ pydantic/color.py:159: error: Cannot determine type of "li"  [has-type]
+ pydantic/color.py:159: error: Cannot determine type of "h"  [has-type]
+ pydantic/color.py:162: error: Cannot determine type of "s"  [has-type]
+ pydantic/color.py:162: error: Cannot determine type of "li"  [has-type]
+ pydantic/color.py:162: error: Cannot determine type of "h"  [has-type]
+ pydantic/color.py:162: error: Cannot determine type of "a"  [has-type]

Maybe it's better to prevent such follow-up error reports by analysing all possible union members instead of quitting when a single union member is problematic.

The intermediate construction of tuples is unnecessary. Checking each lvalue directly against the union of its relevant rvalue types is sufficient and avoids the hard-to-understand convert_star_rvalue_type argument.

I will change both aspects in the next commit.

Diff from mypy_primer, showing the effect of this PR on open source code:

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/test.py:1273: error: Unused "type: ignore" comment
+ src/werkzeug/routing/map.py:734: error: Unused "type: ignore" comment

porcupine (https://github.com/Akuli/porcupine)
+ porcupine/plugins/directory_tree.py:412: error: Unused "type: ignore" comment

Seems, Mypy primer tells me I forgot to handle other kinds of not iterable types like None and Literal.

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

@ilevkivskyi: Would you like to review this one? I am asking you because it changes code you worked on. (And because this pull request grows old. So far, resolving conflicts should be simple.)