Provide context on why an assignment failed

[dhruvmanila]

It will be useful to provide why an assignment between two types failed. Pyright does this but for ty it will require updating the APIs because currently the returned type is boolean for is_assignable_to.

[MatthewMckee4]

Would we be looking to create an enum to represent a reason for a lack of assignability/subtyping?

Then return

enum RelationResult {
    Pass
    Fail(Reason)
}

[AlexWaygood]

Possibly has_relation_to could return Result<(), AssignabilityError>:

• If the first type is a subtype of (or assignable to) the second type, Ok(()) is returned
• If the first type doesn't have the given relation to the second type, however, Err(Assignability) is returned, where AssignabilityError is an enum that provides context on why the relation doesn't exist

[MatthewMckee4]

Sounds good.

Just wondering but what is the benefit of using a result for this. I think I'm right in saying Result<(), ...> is equivalent to Option<...> and also to my example. What makes you choose any one of these?

[MatthewMckee4]

Also do we not want the error type to represent subtyping issues too?

[AlexWaygood]

Just wondering but what is the benefit of using a result for this. I think I'm right in saying Result<(), ...> is equivalent to Option<...> and also to my example. What makes you choose any one of these?

I don't think using an Option or a "top-level" enum would be wrong -- but if an operation could succeed or fail (and we want to provide context for why it failed in the failure case), it seems natural to me to use a Result to represent that. It's the type that is generally used for this kind of situation, and it has language support built around it (e.g. you can use the ? operator to propagate errors upwards, and map_err to convert one kind of error into another).

Also do we not want the error type to represent subtyping issues too?

Yes, good point! TypeRelationError might be a better name.

[MatthewMckee4]

Okay sounds good, I'd be happy to set this up, it seems like we need a sort of base case for the enum (or a todo variant, or both?) to start.

[MatthewMckee4]

Making a small start on this. It seems fair to be able to allow multiple reasons, for union types for example.

What do we think of this as a base for TypeRelationError?

#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) struct TypeRelationError(pub(crate) Vec<TypeRelationErrorKind>);

impl TypeRelationError {
    pub(crate) fn single(kind: TypeRelationErrorKind) -> Self {
        Self(vec![kind])
    }
}

#[derive(Debug, Copy, Clone, PartialEq, Eq)]
pub(crate) enum TypeRelationErrorKind {
    Todo,
    GradualTypeInSubTyping,
}

[AlexWaygood]

That looks like a reasonable design to me!

[MatthewMckee4]

Should this also cover is_equivalent_to functions?

[AlexWaygood]

Maybe as a followup, but I think this task in itself will involve some refactoring, so I wouldn't expand the scope for the initial PR

[jelle-openai]

Possibly has_relation_to could return Result<(), AssignabilityError>:

• If the first type is a subtype of (or assignable to) the second type, Ok(()) is returned
• If the first type doesn't have the given relation to the second type, however, Err(Assignability) is returned, where AssignabilityError is an enum that provides context on why the relation doesn't exist

For what it's worth, this is quite similar to what I do in pycroscope, except that the error case is nested (code here).

The relevant function returns either:

• A dictionary providing bounds on TypeVars that need to apply for the assignment to work
• Or a CanAssignError object, which contains a string plus a list of deeper CanAssignError objects

The latter gets displayed with nested indented lists:

In [8]: typ = pycroscope.annotations.type_from_runtime(tuple[str, int] | tuple[str, float])

In [10]: typ.can_assign(pycroscope.value.KnownValue((1, "x")), pycroscope.runtime._get_checker())
Out[10]: CanAssignError(message="Literal[(1, 'x')] is not assignable to tuple[str, int] | tuple[str, float]", children=[CanAssignError(message='Elements at position 0 are not compatible', children=[CanAssignError(message='Cannot assign Literal[1] to str', children=[], error_code=None)], error_code=None), CanAssignError(message='Elements at position 0 are not compatible', children=[CanAssignError(message='Cannot assign Literal[1] to str', children=[], error_code=None)], error_code=None)], error_code=None)

In [11]: print(_)
  Literal[(1, 'x')] is not assignable to tuple[str, int] | tuple[str, float]
    Elements at position 0 are not compatible
      Cannot assign Literal[1] to str
    Elements at position 0 are not compatible
      Cannot assign Literal[1] to str

(Maybe not a great example because it doesn't tell you which arm of the union corresponds to the indented parts. I'm sure you can do better there :). But I think the idea of building up a tree of reasons for why the assignment didn't work is useful.)

[AlexWaygood]

If we consider the example I gave in https://github.com/astral-sh/ty/issues/866:

from typing import Callable, Any

def f(x: Callable[[Any], Any]): ...

class Foo:
    def __init__(self, x, y): ...

f(Foo)

Mypy's diagnostic also isn't great in this case:

error: Argument 1 to "f" has incompatible type "type[Foo]"; expected "Callable[[Any], Any]"  [arg-type]

https://mypy-play.net/?mypy=latest&python=3.12&gist=ee7c1164dc400825966268856211fa15

But pyright's is much better here ('Extra parameter "y"'):

Argument of type "type[Foo]" cannot be assigned to parameter "x" of type "(Any) -> Any" in function "f"
  Type "type[Foo]" is not assignable to type "(Any) -> Any"
    Extra parameter "y"  (reportArgumentType)

https://pyright-play.net/?pythonVersion=3.13&strict=true&enableExperimentalFeatures=true&code=GYJw9gtgBALgngBwJYDsDmUkQWEMoDCAhgDYlEBGJApgDRQCCKcAUCwCbXBTAAUAHgC5CpclWoBtCUzgBdejNkBKYQDp1bAMbkAzjqgAxMGEEso5qJ24B9a6iQxbvHdRLB6-enBVR1qtnxGYEosQA

For a similar case involving Protocols, meanwhile, mypy's diagnostic is superior IMO:

from typing import Protocol, Any

class Bar(Protocol):
    def __call__(self, x: Any) -> Any: ...
    y: int

def f(x: Bar): ...

class Foo:
    def __call__(self, x: Any, y: Any) -> Any: ...
    y: int

f(Foo())

Mypy says:

main.py:13: error: Argument 1 to "f" has incompatible type "Foo"; expected "Bar"  [arg-type]
main.py:13: note: Following member(s) of "Foo" have conflicts:
main.py:13: note:     Expected:
main.py:13: note:         def __call__(self, x: Any) -> Any
main.py:13: note:     Got:
main.py:13: note:         def __call__(self, x: Any, y: Any) -> Any
main.py:13: note: "Bar.__call__" has type "Callable[[Arg(Any, 'x')], Any]"

Pyright says:

Argument of type "Foo" cannot be assigned to parameter "x" of type "Bar" in function "f"
  "Foo" is incompatible with protocol "Bar"
    "__call__" is an incompatible type
      Type "(x: Any, y: Any) -> Any" is not assignable to type "(x: Any) -> Any"
        Extra parameter "y"  (reportArgumentType)

[carljm]

https://github.com/astral-sh/ty/issues/1591 is another example case to consider here, where pyright provides a fantastic diagnostic.

Pyrefly also does alright here -- it doesn't provide the full chain like pyright does, but it gives you the inner-most detail.