mypy icon indicating copy to clipboard operation
mypy copied to clipboard

Type is incompatible with Callable when using Concatenate

Open ringohoffman opened this issue 3 years ago • 1 comments

Bug Report

As expected, a class A can be typed as Type[A] or Callable[P, A], where P is a ParamSpec.

Using Concatenate then, you should also be able to bind A to Callable[Concatenate[<type>, P], A], where <type> is the type of A's first constructor parameter, but mypy currently errors when you attempt this.

To Reproduce

gist: https://gist.github.com/mypy-play/1cab71a52f3d7f3a4e9045e6e68eabe0 mypy-playground: https://mypy-play.net/?mypy=latest&python=3.10&gist=1cab71a52f3d7f3a4e9045e6e68eabe0

from __future__ import annotations

from typing import Callable, Concatenate, ParamSpec

P = ParamSpec("P")


class A:
    def __init__(self, a_param_1: str) -> None:
        ...

    @classmethod
    def add_params(cls: Callable[P, A]) -> Callable[Concatenate[float, P], A]:
        def new_constructor(i: float, *args: P.args, **kwargs: P.kwargs) -> A:
            return cls(*args, **kwargs)
        return new_constructor
    
    @classmethod
    def remove_params(cls: Callable[Concatenate[str, P], A]) -> Callable[P, A]:
        def new_constructor(*args: P.args, **kwargs: P.kwargs) -> A:
            return cls("my_special_str", *args, **kwargs)
        return new_constructor

A.add_params()  # OK
A.remove_params()  # [misc] mypy(error): Invalid self argument "Type[A]" to attribute function "remove_params" with type "Callable[[Callable[[str, **P], A]], Callable[P, A]]"

Expected Behavior

No error should be raised.

Actual Behavior

error: Invalid self argument "Type[A]" to attribute function "remove_params" with type "Callable[[Callable[[str, **P], A]], Callable[P, A]]"  [misc]

Your Environment

  • Mypy version used: mypy 0.991 (compiled: yes)
  • Python version used: 3.10.6

ringohoffman avatar Dec 20 '22 08:12 ringohoffman

The difference in behavior between add_params and remove_params seems to come from this line in check_self_arg :

if subtypes.is_subtype(dispatched_arg_type, erase_typevars(erase_to_bound(selfarg))):
name selfarg dispatched_arg_type subtypes.is_subtype(...)
add_params def (*P.args, **P.kwargs) -> A def (a_param_1: builtins.str) -> A True from are_trivial_parameters(...) in are_parameters_compatible
remove_params def (builtins.str, *P.args, **P.kwargs) -> A def (a_param_1: builtins.str) -> A False from not allow_partial_overlap in are_parameters_compatible

In the case of remove_params, by setting is_callable_compatible(allow_partial_overlap=right.from_concatenate or left.from_concatenate) in visit_callable_type, we can progress a little further into are_parameters_compatible... however we then return False from the condition in "Phase 2":

if (
    right_by_name is not None
    and right_by_pos is not None
    and right_by_name != right_by_pos
    and (right_by_pos.required or right_by_name.required)
    and strict_concatenate_check
):
    return False
right_by_name right_by_pos
FormalArgument(name='a_param_1', pos=None, typ=Any, required=False) FormalArgument(name=None, pos=0, typ=builtins.str, required=True)

I feel like if def (a_param_1: builtins.str) -> A is considered a subtype of def (*P.args, **P.kwargs) -> A, then it should also be a subtype of def (builtins.str, *P.args, **P.kwargs) -> A, since this seems to be how Concatenate works in similar constructions:

from typing import ParamSpec, TypeVar

P = ParamSpec("P")
T = TypeVar("T")


def decorator(f: Callable[Concatenate[str, P], T]) -> Callable[P, T]:
    def new_func(*args: P.args, **kwargs: P.kwargs) -> T:
        return f("my_special_str", *args, **kwargs)
    return new_func


@decorator  # OK
def foo(foo_param_1: str) -> None:
    ...

@A5rocks @cdce8p @JukkaL Does my reasoning and proposed change seem sound? What change related to the "Phase 2" condition should be made s.t. the subtype check passes?

ringohoffman avatar Dec 29 '22 09:12 ringohoffman