Correct type hints for meta-function that takes a function as an argument (ParamSpec)

[MattRCole]

Question:

Let's say we have the following function:

def not_a_decorator(fn, /, *args, **kwargs):
    # do some stuff
    return fn(*args, **kwargs)

How would we go about typing not_a_decorator so that pylance can dynamically tell us what args and kwargs are expected based on the given fn?

Example:

def not_a_decorator(fn, *args, **kwargs):
    # do some stuff
    return fn(*args, **kwargs)
    
def foo(arg1: int, /,  arg2: str, *, arg3: float) -> bool:
    return arg1 == 1 and arg2 == 'hi' and arg3 == 0.0
    
result = not_a_decorator(foo, 1, arg2='hi', arg3=0.0)  # How do we get type hints for result and the parameters of not_a_decorator?

Why do this?

Although this seems like an unusual usecase, it's not too far fetched. Take functools.partial: While the return value of functools.partial is very hard to type, it would be useful to have type hints for the correct parameters to pass to partial based on the function that was passed in as the first argument.

My attempted solutions

Typing the function directly

from typing import TypeVar, ParamSpec, Callable


V = TypeVar('V')
P = ParamSpec('P')

def not_a_decorator(fn: Callable[P, V], /, *args: P.args, **kwargs: P.kwargs) -> V:
    # do some stuff right here
    return fn(*args, **kwargs)


def foo(arg1: int, /, arg2: str, *, arg3: float) -> bool:
    return arg1 == 1 and arg2 == 'hi' and arg3 == 0.0


result = not_a_decorator(
    foo,
    ..., # args go here
)

This fails to provide good parameter type hints:

Making a type alias and using Concatenate

from typing import TypeVar, ParamSpec, Callable, Concatenate


V = TypeVar('V')
P = ParamSpec('P')


NotADecorator = Callable[Concatenate[Callable[P, V], P], V]

not_a_decorator: NotADecorator = lambda fn, *args, **kwargs: fn(*args, **kwargs)

def foo(arg1: int, /, arg2: str, *, arg3: float) -> bool:
    return arg1 == 1 and arg2 == 'hi' and arg3 == 0.0


not_a_decorator(
    foo,
    ..., # parameters go here
)

This also fails to provide good parameter type hints:

Using a Protocol

This was the final way I could think of to achieve this

from typing import TypeVar, ParamSpec, Callable, Protocol


V = TypeVar('V')
P = ParamSpec('P')


class NotADecorator(Protocol):
    def __call__(
        _,
        fn: Callable[P, V],
        /,
        *args: P.args,
        **kwargs: P.kwargs,
    ) -> V:
        ...

not_a_decorator: NotADecorator = lambda fn, *args, **kwargs: fn(*args, **kwargs)

def foo(arg1: int, /, arg2: str, *, arg3: float) -> bool:
    return arg1 == 1 and arg2 == 'hi' and arg3 == 0.0


not_a_decorator(
    foo,
    ..., # parameters go here
)

Unfortunately, this also fails in the same manner:

Conclusion

It really feels like ParamSpec should allow this kind of dynamic type hinting, but it doesn't, or at least not in these three ways. Is there a different way to do this that I missed or is this not currently supported? If not, is it in-scope to support such a use-case of ParamSpec?

[erictraut]

ParamSpec is the right way to annotate a function like not_a_decorator in your example. Your instincts were spot on here. If you enable type checking (set "typeCheckingMode" to "basic"), you can see that pyright (the type checker upon which pylance is built) will validate the arguments passed to not_a_decorator and report type violations if present.

What you're looking for here is a language server feature — the ability for the signature help provider to have an understanding of parameters that are provided by an earlier argument that is bound to a ParamSpec. I don't recall anyone requesting this feature previously, but it's a reasonable enhancement request for pylance.

[bcb2020]

what does mean triage-needed?

[debonte]

what does mean triage-needed?

It means that the Pylance dev team needs to discuss how to address the issue.

[aabdagic]

jfyi, this seems to be working already:

Given:

class Base:
    @classmethod
    def create(
        cls: Callable[P, T], *args: P.args, **kwargs: P.kwargs
    ) -> T:
        return cls(*args, **kwargs)


class Derived(Base):
    def __init__(self, foo: str):
        self._foo = foo

I see the right completion on Derived.create.

[hemildesai]

+1 to this feature. It would be great to have autocomplete working for ParamSpec

[MrSamuelLaw]

Below is my attempt to show the expected parameters from partial. In Pyrefly it expands out the signatures of P & Q to correctly provide accurate auto complete and type hinting. Pylance on the other hand fails to carry over the signatures of P & Q and instead just shows a literal "P" and a literal "Q". I would just use Pyrefly, but alas, it does not yet play nice with Pydantic.

from typing import (
    TypeVar, 
    Callable, 
    Protocol, 
    runtime_checkable
)
from dataclasses import dataclass
from functools import partial as _partial





T = TypeVar("T")
U = TypeVar("U")

@runtime_checkable
class Taggable[T](Protocol):

    @staticmethod
    def __call__(*, tag: str, description: str) -> T:...

def typed_partial[**P, **Q](cls: type[T], rtn: type[U]) -> Callable[P, Callable[Q, U]]:
    def wrapper(*args: P.args, **kwargs: P.kwargs) -> Callable[Q, U]:
        return _partial(cls, *args, **kwargs)
    return wrapper


@dataclass(kw_only=True)
class Foo:
    a: int
    tag: str = None
    description: str = None

x = typed_partial(Foo, Taggable)