basedmypy icon indicating copy to clipboard operation
basedmypy copied to clipboard

Tighten overload impl return type

Open KotlinIsland opened this issue 3 years ago • 5 comments

@overload
def foo(a: int) -> int: ...
@overload
def foo(a: str) -> str: ...
def foo(a: object) -> object:
  return object()

I think that the return type shouldn't be completely unchecked, it should be a union of the overloads return types or narrower.

The true form of the return type isn't an intersection, it's actually wider than that. It's a conditional type based on the overloads.

KotlinIsland avatar Apr 04 '22 03:04 KotlinIsland

The true form of the return type isn't an intersection, ...

Shouldn't that intersection be union (as in python int and str do not overlap)?

Zeckie avatar Apr 24 '22 10:04 Zeckie

Could basedmypy detect that a function cannot return a type?

eg. this definition of foo never returns a str

@overload
def foo(a: int) -> int: ...
@overload
def foo(a: str) -> str: ...
def foo(a: object) -> str | int:
  return 1

Zeckie avatar Apr 24 '22 10:04 Zeckie

@Zeckie

Shouldn't that intersection be union

An overload's return type is typesafe as an intersection of the signatures rather than a union (which is unsafe):

class A:
    a: int
class B: 
    b: int
@overload
def foo(x: A) -> A: ...
@overload
def foo(x: B) -> B: ...

# foo or foos return type is not a union of the overloads:
def f1(a: A) -> A:
    print(a.a)
    return a
foo1: (x: A) -> A | (x: B) -> B = f1
foo1(B())  # error, B object has no attribute 

def f2(a: A | B) -> A | B:
    return A()
foo2: (x: A | B) -> A | B = f2
foo2(B()).b  # error, A object has no attribute b

# The actual return type is an intersection of the overloads return types: `(x: A) -> A & (x: B) -> B` or `(x: A | B) -> A & B`.
def f3(x: A | B) -> A & B:
    class C(A, B): ...
    return C()

foo3: (x: A | B) -> A & B = f3
res = foo3(A())
res.a, res.b  # no error

Which in the case of int and str results in Never because they have an incompatible slot layout. Although the true form of an overload is not an intersection but is actually a bit wider than that (yet still narrower than a union).

Could basedmypy detect that a function cannot return a type?

see #245

KotlinIsland avatar Apr 24 '22 11:04 KotlinIsland

Isn't it that the return type needs to include (eg. using a Union) a type that is the same as, or overlaps the return type of each overload (it has to be able to return something that matches each overload, but can return anything that matches either overload). The intersection would only be required in cases where the function needs to return the same type, no matter what the input.

In your example, foo2 is declared to return A | B but actually only returns A. Another implementation of that function with the same signature could be fine (return A when param is an A, B when param is a B).

https://mypy-play.net/?mypy=latest&python=3.10&gist=2431dc358a99933e7934feec2e800665

Could basedmypy detect that a function cannot return a type?

My thought was that basedmypy could validate the implementing function using the overload's signature (with some modifications to handle cases where overload doesn't include some arguments with default values), to make sure that when a is a string, the return is a string (or at least not something that is definitely incompatible, such as an int).

Zeckie avatar Apr 25 '22 23:04 Zeckie

Isn't it that the return type needs to include (eg. using a Union) a type that is the same as, or overlaps the return type of each overload

That isn't type safe: (int) -> int & (str) -> str != (int | str) -> int | str

@overload
def foo(a: int) -> int: ...
@overload
def foo(a: str) -> str: ...
def foo(a: int | str) -> int | str:
    return 1

s: str = foo("")

An intersection of the overload signature's return types will ensure type safety (but is overly strict).

In your example, foo2 is declared to return A | B but actually only returns A. Another implementation of that function with the same signature could be fine (return A when param is an A, B when param is a B).

That is correct, but it's the same as saying:

@overload
def foo() -> Literal[1]: ...
def foo() -> object:
    return 1

The impl could return the correct type, but it's clear that the signature here is incorrect.

My thought was that basedmypy could validate the implementing function

That is what #245 covers

KotlinIsland avatar Apr 26 '22 02:04 KotlinIsland