False positive `reportAssignmentType` with `list.__add__` type widening from a newly-created object

[Phrogz]

Description

foo: list[str] = ["a", "b"]
bar: list[str | float] = foo + ["c"]     # incorrectly shows a type-widening error here
baz: list[str | float] = ["a", "b", "c"] # type-widening is correctly accepted here
bat: list[str | float] = foo             # correctly shows a type-widening error here

The bar line should be treated as the baz, as list + list creates a new list with no other references to it; mutating bar does not/cannot affect foo.

Link to BasedPyright Playground Example

Occurs in Pyright as well; filed upstream as Pyright bug #10943.

[KotlinIsland]

hmm, is this a typeshed issue?

[beauxq]

I think it would require extra steps of inference (likely a performance limitation) or some kind of ownership model in the type system to know that there are no other references to the value returned by list.__add__

[KotlinIsland]

can you provide an example of this being unsound? as far as I can think:

class list[T]:
    def __add__[O](self, other: list[O]) -> list[T | O]:
        result = []
        result.extend(self)
        result.extend(other)
        return result

this seems valid to me

[beauxq]

I don't think that's unsound. But how are we supposed to get the inference of the type variables that we need from the code in the OP?

[KotlinIsland]

I haven't investigated the signature yet, but there is no issue inferring other as list[float]

Code sample in basedpyright playground

[beauxq]

I haven't investigated the signature yet, but there is no issue inferring other as list[float]

The OP has ["c"] in the position that would need to be list[float]

[KotlinIsland]

ah, my mistake, it's still normal bidirectional type inference, should be fine

Code sample in basedpyright playground

[KotlinIsland]

    # Overloading looks unnecessary, but is needed to work around complex mypy problems
    @overload
    def __add__(self, value: list[_T], /) -> list[_T]: ...
    @overload
    def __add__(self, value: list[_S], /) -> list[_S | _T]: ...

so this is a nonsense mypy issue, i think we update the vendored typeshed and see what comes out of the primer

[beauxq]

ah, my mistake, it's still normal bidirectional type inference, should be fine

Code sample in basedpyright playground

That sample is covariant.

This one is invariant: basedpyright playground

[DetachHead]

i think this needs more than just a typeshed update. there seems to be some special casing when using the + operator that results in different behavior to when the dunder is called like a regular method:

-     # Overloading looks unnecessary, but is needed to work around complex mypy problems
-     @overload
-     def __add__(self, value: list[_T], /) -> list[_T]: ...
-     @overload
-     def __add__(self, value: list[_S], /) -> list[_S | _T]: ...
+     def __add__(self, value: list[_S | _T], /) -> list[_S | _T]:
        """Return self+value."""
        ...

foo: list[str] = ["a", "b"]
bar: list[str | float] = foo.__add__(["c"])  # now works
baz: list[str | float] = foo + ["c"] # still errors

[KotlinIsland]

i fixed it: Code sample in basedpyright playground

we need to capture the expected type as well

[KotlinIsland]

extreme sus detected: Code sample in basedpyright playground

looks like the order of the union is affecting the behaviour

• #1492

[DetachHead]

i think this needs more than just a typeshed update. there seems to be some special casing when using the + operator that results in different behavior to when the dunder is called like a regular method:

-     # Overloading looks unnecessary, but is needed to work around complex mypy problems
-     @overload
-     def __add__(self, value: list[_T], /) -> list[_T]: ...
-     @overload
-     def __add__(self, value: list[_S], /) -> list[_S | _T]: ...
+     def __add__(self, value: list[_S | _T], /) -> list[_S | _T]:
        """Return self+value."""
        ...

foo: list[str] = ["a", "b"]
bar: list[str | float] = foo.__add__(["c"])  # now works
baz: list[str | float] = foo + ["c"] # still errors

#1493