Updated functools.wraps and functools.update_wrapper to use ParamSpec…

[erictraut]

… to preserve signatures of wrapped and wrapper functions.

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

jax (https://github.com/google/jax)
+ jax/example_libraries/optimizers.py:206: error: Incompatible return value type (got "_Wrapped[Callable[..., None], Tuple[Callable[[Any], Any], Callable[[int, Any, Any], Any], Callable[[Any], Any]], Callable[[VarArg(Any), KwArg(Any)], None], Any]", expected "Callable[..., Optimizer]")  [return-value]
+ jax/example_libraries/optimizers.py:206: note: "_Wrapped[Callable[..., None], Tuple[Callable[[Any], Any], Callable[[int, Any, Any], Any], Callable[[Any], Any]], Callable[[VarArg(Any), KwArg(Any)], None], Any].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Tuple[Callable[[Any], Any], Callable[[int, Any, Any], Any], Callable[[Any], Any]]]"

zulip (https://github.com/zulip/zulip)
+ zerver/lib/type_debug.py:75: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/profile.py:27: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/cache.py:143: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/cache.py:176: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/openapi/python_examples.py:44: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ corporate/lib/stripe.py:247: error: Argument 1 to "wraps" has incompatible type "CallableT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/decorator.py:871: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/tests/test_service_bot_system.py:419: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]

rich (https://github.com/willmcgugan/rich)
+ rich/console.py:498: error: Incompatible return value type (got "_Wrapped[Callable[..., None], Iterable[Union[ConsoleRenderable, RichCast, str]], Callable[[VarArg(Any), KwArg(Any)], None], Group]", expected "Callable[..., Group]")
+ rich/console.py:498: note: "_Wrapped[Callable[..., None], Iterable[Union[ConsoleRenderable, RichCast, str]], Callable[[VarArg(Any), KwArg(Any)], None], Group].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Iterable[Union[ConsoleRenderable, RichCast, str]]]"

pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/fixtures.py:1134: error: Argument 1 to "wraps" has incompatible type "FixtureFunction"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]

bokeh (https://github.com/bokeh/bokeh)
+ release/credentials.py: note: In function "collect_credential":
+ release/credentials.py:61:16: error: Incompatible return value type (got "_Wrapped[Callable[..., None], None, Callable[[Config, System], None], ActionReturn]", expected "Callable[[Config, System], ActionReturn]")  [return-value]
+ release/credentials.py:61:16: note: "_Wrapped[Callable[..., None], None, Callable[[Config, System], None], ActionReturn].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], None]"
+ bokeh/document/callbacks.py: note: In member "add_session_callback" of class "DocumentCallbackManager":
+ bokeh/document/callbacks.py:182:31: error: Incompatible types in assignment (expression has type "Callable[[], None]", variable has type "_Wrapped[Callable[[], None], None, Callable[[], None], None]")  [assignment]

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/wsgi.py:34: error: Incompatible return value type (got "_Wrapped[Callable[..., None], Callable[[Dict[str, Any], StartResponse], Iterable[bytes]], Callable[[VarArg(Any)], None], Iterable[bytes]]", expected "Callable[[Dict[str, Any], StartResponse], Iterable[bytes]]")  [return-value]
+ src/werkzeug/wsgi.py:34: note: "_Wrapped[Callable[..., None], Callable[[Dict[str, Any], StartResponse], Iterable[bytes]], Callable[[VarArg(Any)], None], Iterable[bytes]].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Callable[[Dict[str, Any], StartResponse], Iterable[bytes]]]"
+ src/werkzeug/formparser.py:153: error: Incompatible return value type (got "_Wrapped[<nothing>, <nothing>, <nothing>, <nothing>]", expected "F")  [return-value]

pandas (https://github.com/pandas-dev/pandas)
+ pandas/util/_decorators.py:90: error: Unused "type: ignore" comment
+ pandas/core/strings/accessor.py:127: error: "_Wrapped[<nothing>, <nothing>, Callable[[Any, VarArg(Any), KwArg(Any)], None], Any]" has no attribute "__name__"  [attr-defined]

starlette (https://github.com/encode/starlette)
+ starlette/routing.py:513: error: Incompatible return value type (got "_Wrapped[Callable[[Any], None], _GeneratorContextManager[Any], Callable[[Any], None], _AsyncLiftContextManager[Any]]", expected "Callable[[Any], AsyncContextManager[Any]]")  [return-value]
+ starlette/routing.py:513: note: "_Wrapped[Callable[[Any], None], _GeneratorContextManager[Any], Callable[[Any], None], _AsyncLiftContextManager[Any]].__call__" has type "Callable[[Any], _GeneratorContextManager[Any]]"

mkosi (https://github.com/systemd/mkosi)
+ mkosi/__init__.py:169:12: error: Incompatible return value type (got "_Wrapped[Callable[..., None], Iterator[Tuple[T, V]], Callable[[VarArg(Any), KwArg(Any)], None], Dict[T, V]]", expected "Callable[..., Dict[T, V]]")  [return-value]
+ mkosi/__init__.py:169:12: note: "_Wrapped[Callable[..., None], Iterator[Tuple[T, V]], Callable[[VarArg(Any), KwArg(Any)], None], Dict[T, V]].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Iterator[Tuple[T, V]]]"

ibis (https://github.com/ibis-project/ibis)
- ibis/backends/impala/hdfs.py:464: error: Return type "Dict[Any, Any]" of "exists" incompatible with return type "bool" in supertype "HDFS"
+ ibis/backends/impala/hdfs.py:529: error: Missing positional argument "path" in call to "__call__" of "_Wrapped"
+ ibis/backends/impala/hdfs.py:529: error: Argument 1 to "__call__" of "_Wrapped" has incompatible type "str"; expected "HDFS"
- ibis/backends/impala/hdfs.py:615: error: Signature of "get" incompatible with supertype "HDFS"
- ibis/backends/impala/hdfs.py:615: note:      Superclass:
- ibis/backends/impala/hdfs.py:615: note:          def get(self, hdfs_path: str, local_path: str = ..., overwrite: bool = ...) -> str
- ibis/backends/impala/hdfs.py:615: note:      Subclass:
- ibis/backends/impala/hdfs.py:615: note:          def get(self, hdfs_path: str, local_path: str, overwrite: bool = ..., verbose: Optional[bool] = ..., **kwargs: Any) -> str

spark (https://github.com/apache/spark)
+ python/pyspark/sql/udf.py:278: error: "_Wrapped[Callable[..., None], Any, Callable[[VarArg(Union[Column, str])], None], Column]" has no attribute "__name__"  [attr-defined]
+ python/pyspark/pandas/base.py:252: error: Incompatible return value type (got "_Wrapped[Callable[..., None], Column, Callable[[Union[Series[Any], Index], VarArg(Any)], None], Union[Series[Any], Index]]", expected "Callable[..., Union[Series[Any], Index]]")  [return-value]
+ python/pyspark/pandas/base.py:252: note: "_Wrapped[Callable[..., None], Column, Callable[[Union[Series[Any], Index], VarArg(Any)], None], Union[Series[Any], Index]].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Column]"
+ python/pyspark/pandas/base.py:269: error: Incompatible return value type (got "_Wrapped[Callable[..., None], Column, Callable[[Union[Series[Any], Index], VarArg(Any)], None], Union[Series[Any], Index]]", expected "Callable[..., Union[Series[Any], Index]]")  [return-value]
+ python/pyspark/pandas/base.py:269: note: "_Wrapped[Callable[..., None], Column, Callable[[Union[Series[Any], Index], VarArg(Any)], None], Union[Series[Any], Index]].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Column]"

steam.py (https://github.com/Gobot1234/steam.py)
+ steam/utils.py:539: error: Incompatible return value type (got "_Wrapped[Callable[_P, None], Awaitable[None], Callable[_P, None], Coroutine[Any, Any, None]]", expected "Callable[_P, Coroutine[Any, Any, None]]")  [return-value]
+ steam/utils.py:539: note: "_Wrapped[Callable[_P, None], Awaitable[None], Callable[_P, None], Coroutine[Any, Any, None]].__call__" has type "Callable[_P, Awaitable[None]]"

tornado (https://github.com/tornadoweb/tornado)
+ tornado/gen.py:270: error: Incompatible return value type (got "_Wrapped[Callable[..., None], object, Callable[[VarArg(Any), KwArg(Any)], None], Future[_T]]", expected "Callable[..., Future[_T]]")
+ tornado/gen.py:270: note: "_Wrapped[Callable[..., None], object, Callable[[VarArg(Any), KwArg(Any)], None], Future[_T]].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], object]"
- tornado/testing.py:595: error: Incompatible types in assignment (expression has type "Callable[..., Future[Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]]", variable has type "Callable[[AsyncTestCase, VarArg(Any), KwArg(Any)], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]")
+ tornado/testing.py:595: error: Incompatible types in assignment (expression has type "Callable[..., Future[Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]]", variable has type "_Wrapped[Callable[..., None], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]], Callable[[AsyncTestCase, VarArg(Any), KwArg(Any)], None], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]")
+ tornado/testing.py:621: error: Incompatible return value type (got "_Wrapped[Callable[[VarArg(Any), KwArg(Any)], None], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]], Callable[[AsyncTestCase, VarArg(Any), KwArg(Any)], None], None]", expected "Callable[..., None]")
+ tornado/testing.py:621: note: "_Wrapped[Callable[[VarArg(Any), KwArg(Any)], None], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]], Callable[[AsyncTestCase, VarArg(Any), KwArg(Any)], None], None].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]"

core (https://github.com/home-assistant/core)
+ homeassistant/auth/permissions/util.py:70: error: Incompatible return value type (got "_Wrapped[Callable[[str, str], None], Optional[bool], Callable[[str, str], None], bool]", expected "Callable[[str, str], bool]")  [return-value]
+ homeassistant/auth/permissions/util.py:70: note: "_Wrapped[Callable[[str, str], None], Optional[bool], Callable[[str, str], None], bool].__call__" has type "Callable[[str, str], Optional[bool]]"
+ homeassistant/components/websocket_api/decorators.py:47: error: Incompatible return value type (got "_Wrapped[Callable[[HomeAssistant, ActiveConnection, Dict[str, Any]], None], Awaitable[None], Callable[[HomeAssistant, ActiveConnection, Dict[str, Any]], None], None]", expected "Callable[[HomeAssistant, ActiveConnection, Dict[str, Any]], None]")  [return-value]
+ homeassistant/components/websocket_api/decorators.py:47: note: "_Wrapped[Callable[[HomeAssistant, ActiveConnection, Dict[str, Any]], None], Awaitable[None], Callable[[HomeAssistant, ActiveConnection, Dict[str, Any]], None], None].__call__" has type "Callable[[HomeAssistant, ActiveConnection, Dict[str, Any]], Awaitable[None]]"
+ homeassistant/components/mqtt/__init__.py:378: error: Incompatible types in assignment (expression has type "_Wrapped[Callable[[str, Union[str, bytes], int], None], Optional[Awaitable[None]], Callable[[ReceiveMessage], None], Coroutine[Any, Any, None]]", variable has type "Union[Callable[[ReceiveMessage], Awaitable[None]], Callable[[ReceiveMessage], None]]")  [assignment]
+ homeassistant/components/mqtt/__init__.py:386: error: Incompatible types in assignment (expression has type "_Wrapped[Callable[[str, Union[str, bytes], int], None], Optional[Awaitable[None]], Callable[[ReceiveMessage], None], None]", variable has type "Union[Callable[[ReceiveMessage], Awaitable[None]], Callable[[ReceiveMessage], None]]")  [assignment]

boostedblob (https://github.com/hauntsaninja/boostedblob)
+ boostedblob/cli.py:30: error: Incompatible return value type (got "_Wrapped[Callable[..., None], Awaitable[T], Callable[[VarArg(Any), KwArg(Any)], None], Any]", expected "Callable[..., T]")
+ boostedblob/cli.py:30: note: "_Wrapped[Callable[..., None], Awaitable[T], Callable[[VarArg(Any), KwArg(Any)], None], Any].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Awaitable[T]]"

kopf (https://github.com/nolar/kopf)
+ kopf/_kits/webhacks.py:80: error: Argument 1 to "wraps" has incompatible type "_ServerFn"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"

pip (https://github.com/pypa/pip)
+ src/pip/_internal/index/collector.py:293: error: Incompatible return value type (got "_Wrapped[Callable[[HTMLPage], None], Iterable[Link], Callable[[HTMLPage], None], List[Link]]", expected "Callable[[HTMLPage], List[Link]]")
+ src/pip/_internal/index/collector.py:293: note: "_Wrapped[Callable[[HTMLPage], None], Iterable[Link], Callable[[HTMLPage], None], List[Link]].__call__" has type "Callable[[HTMLPage], Iterable[Link]]"

[erictraut]

I'm not sure how to make the pytype CI script happy in this case. In fact, it's not obvious to me from the output what it's complaining about. Any suggestions? @JelleZijlstra? @srittau?

[srittau]

This is a pytype issue. Maybe @rchen152 has some insight.

[rchen152]

This is a pytype issue. Maybe @rchen152 has some insight.

Huh, that's a strange error. It usually occurs when some name is used twice in a stub (e.g.,

x: int
x: str

) but that's clearly not the case here. I'll take a look.

FYI we're not doing any more pytype releases until after the New Year, since most of the team is OOO, so it'll take a little longer than usually to ship a fix :sweat_smile:

[srittau]

One thing to try: As we are effectively "throwing away" the wrapper function's signature (except for the __wrapped__ field) and replacing it by the wrapped function's, we could use a simple _Wrapper = TypeVar("_Wrapper", bound=Callable[..., Any]) type var. _P2 and _R2 are always used together, and _P2.args or _P2.kwargs are never accessed. This could at least help us get a clearer picture of what is going wrong.

[AlexWaygood]

I wonder if inheriting from Protocol rather than Generic might reduce the primer diff somewhat?

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

rich (https://github.com/willmcgugan/rich)
+ rich/console.py:498: error: Incompatible return value type (got "_Wrapped[Callable[..., None], Iterable[Union[ConsoleRenderable, RichCast, str]], Callable[[VarArg(Any), KwArg(Any)], Group]]", expected "Callable[..., Group]")
+ rich/console.py:498: note: "_Wrapped[Callable[..., None], Iterable[Union[ConsoleRenderable, RichCast, str]], Callable[[VarArg(Any), KwArg(Any)], Group]].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Iterable[Union[ConsoleRenderable, RichCast, str]]]"

jax (https://github.com/google/jax)
+ jax/example_libraries/optimizers.py:206: error: Incompatible return value type (got "_Wrapped[Callable[..., None], Tuple[Callable[[Any], Any], Callable[[int, Any, Any], Any], Callable[[Any], Any]], Callable[[VarArg(Any), KwArg(Any)], Any]]", expected "Callable[..., Optimizer]")  [return-value]
+ jax/example_libraries/optimizers.py:206: note: "_Wrapped[Callable[..., None], Tuple[Callable[[Any], Any], Callable[[int, Any, Any], Any], Callable[[Any], Any]], Callable[[VarArg(Any), KwArg(Any)], Any]].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Tuple[Callable[[Any], Any], Callable[[int, Any, Any], Any], Callable[[Any], Any]]]"

pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/fixtures.py:1134: error: Argument 1 to "wraps" has incompatible type "FixtureFunction"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]

zulip (https://github.com/zulip/zulip)
+ zerver/lib/type_debug.py:75: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/profile.py:27: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/cache.py:143: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/cache.py:176: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/openapi/python_examples.py:44: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ corporate/lib/stripe.py:247: error: Argument 1 to "wraps" has incompatible type "CallableT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/decorator.py:871: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/tests/test_service_bot_system.py:419: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]

bokeh (https://github.com/bokeh/bokeh)
+ release/credentials.py: note: In function "collect_credential":
+ release/credentials.py:61:16: error: Incompatible return value type (got "_Wrapped[Callable[..., None], None, Callable[[Config, System], ActionReturn]]", expected "Callable[[Config, System], ActionReturn]")  [return-value]
+ release/credentials.py:61:16: note: "_Wrapped[Callable[..., None], None, Callable[[Config, System], ActionReturn]].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], None]"
+ bokeh/document/callbacks.py: note: In member "add_session_callback" of class "DocumentCallbackManager":
+ bokeh/document/callbacks.py:182:31: error: Incompatible types in assignment (expression has type "Callable[[], None]", variable has type "_Wrapped[Callable[[], None], None, Callable[[], None]]")  [assignment]

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/wsgi.py:34: error: Incompatible return value type (got "_Wrapped[Callable[..., None], Callable[[Dict[str, Any], StartResponse], Iterable[bytes]], Callable[[VarArg(Any)], Iterable[bytes]]]", expected "Callable[[Dict[str, Any], StartResponse], Iterable[bytes]]")  [return-value]
+ src/werkzeug/wsgi.py:34: note: "_Wrapped[Callable[..., None], Callable[[Dict[str, Any], StartResponse], Iterable[bytes]], Callable[[VarArg(Any)], Iterable[bytes]]].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Callable[[Dict[str, Any], StartResponse], Iterable[bytes]]]"
+ src/werkzeug/formparser.py:153: error: Incompatible return value type (got "_Wrapped[<nothing>, <nothing>, F]", expected "F")  [return-value]

pandas (https://github.com/pandas-dev/pandas)
+ pandas/util/_decorators.py:90: error: Unused "type: ignore" comment
+ pandas/core/strings/accessor.py:127: error: "_Wrapped[<nothing>, <nothing>, Callable[[Any, VarArg(Any), KwArg(Any)], Any]]" has no attribute "__name__"  [attr-defined]

mkosi (https://github.com/systemd/mkosi)
+ mkosi/__init__.py:169:12: error: Incompatible return value type (got "_Wrapped[Callable[..., None], Iterator[Tuple[T, V]], Callable[[VarArg(Any), KwArg(Any)], Dict[T, V]]]", expected "Callable[..., Dict[T, V]]")  [return-value]
+ mkosi/__init__.py:169:12: note: "_Wrapped[Callable[..., None], Iterator[Tuple[T, V]], Callable[[VarArg(Any), KwArg(Any)], Dict[T, V]]].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Iterator[Tuple[T, V]]]"

starlette (https://github.com/encode/starlette)
+ starlette/routing.py:513: error: Incompatible return value type (got "_Wrapped[Callable[[Any], None], _GeneratorContextManager[Any], Callable[[Any], _AsyncLiftContextManager[Any]]]", expected "Callable[[Any], AsyncContextManager[Any]]")  [return-value]
+ starlette/routing.py:513: note: "_Wrapped[Callable[[Any], None], _GeneratorContextManager[Any], Callable[[Any], _AsyncLiftContextManager[Any]]].__call__" has type "Callable[[Any], _GeneratorContextManager[Any]]"

steam.py (https://github.com/Gobot1234/steam.py)
+ steam/utils.py:539: error: Incompatible return value type (got "_Wrapped[Callable[_P, None], Awaitable[None], Callable[_P, Coroutine[Any, Any, None]]]", expected "Callable[_P, Coroutine[Any, Any, None]]")  [return-value]
+ steam/utils.py:539: note: "_Wrapped[Callable[_P, None], Awaitable[None], Callable[_P, Coroutine[Any, Any, None]]].__call__" has type "Callable[_P, Awaitable[None]]"

ibis (https://github.com/ibis-project/ibis)
- ibis/backends/impala/hdfs.py:464: error: Return type "Dict[Any, Any]" of "exists" incompatible with return type "bool" in supertype "HDFS"
+ ibis/backends/impala/hdfs.py:529: error: Missing positional argument "path" in call to "__call__" of "_Wrapped"
+ ibis/backends/impala/hdfs.py:529: error: Argument 1 to "__call__" of "_Wrapped" has incompatible type "str"; expected "HDFS"
- ibis/backends/impala/hdfs.py:615: error: Signature of "get" incompatible with supertype "HDFS"
- ibis/backends/impala/hdfs.py:615: note:      Superclass:
- ibis/backends/impala/hdfs.py:615: note:          def get(self, hdfs_path: str, local_path: str = ..., overwrite: bool = ...) -> str
- ibis/backends/impala/hdfs.py:615: note:      Subclass:
- ibis/backends/impala/hdfs.py:615: note:          def get(self, hdfs_path: str, local_path: str, overwrite: bool = ..., verbose: Optional[bool] = ..., **kwargs: Any) -> str

core (https://github.com/home-assistant/core)
+ homeassistant/auth/permissions/util.py:70: error: Incompatible return value type (got "_Wrapped[Callable[[str, str], None], Optional[bool], Callable[[str, str], bool]]", expected "Callable[[str, str], bool]")  [return-value]
+ homeassistant/auth/permissions/util.py:70: note: "_Wrapped[Callable[[str, str], None], Optional[bool], Callable[[str, str], bool]].__call__" has type "Callable[[str, str], Optional[bool]]"
+ homeassistant/components/websocket_api/decorators.py:47: error: Incompatible return value type (got "_Wrapped[Callable[[HomeAssistant, ActiveConnection, Dict[str, Any]], None], Awaitable[None], Callable[[HomeAssistant, ActiveConnection, Dict[str, Any]], None]]", expected "Callable[[HomeAssistant, ActiveConnection, Dict[str, Any]], None]")  [return-value]
+ homeassistant/components/websocket_api/decorators.py:47: note: "_Wrapped[Callable[[HomeAssistant, ActiveConnection, Dict[str, Any]], None], Awaitable[None], Callable[[HomeAssistant, ActiveConnection, Dict[str, Any]], None]].__call__" has type "Callable[[HomeAssistant, ActiveConnection, Dict[str, Any]], Awaitable[None]]"
+ homeassistant/components/mqtt/__init__.py:378: error: Incompatible types in assignment (expression has type "_Wrapped[Callable[[str, Union[str, bytes], int], None], Optional[Awaitable[None]], Callable[[ReceiveMessage], Coroutine[Any, Any, None]]]", variable has type "Union[Callable[[ReceiveMessage], Awaitable[None]], Callable[[ReceiveMessage], None]]")  [assignment]
+ homeassistant/components/mqtt/__init__.py:386: error: Incompatible types in assignment (expression has type "_Wrapped[Callable[[str, Union[str, bytes], int], None], Optional[Awaitable[None]], Callable[[ReceiveMessage], None]]", variable has type "Union[Callable[[ReceiveMessage], Awaitable[None]], Callable[[ReceiveMessage], None]]")  [assignment]

spark (https://github.com/apache/spark)
+ python/pyspark/sql/udf.py:278: error: "_Wrapped[Callable[..., None], Any, Callable[[VarArg(Union[Column, str])], Column]]" has no attribute "__name__"  [attr-defined]
+ python/pyspark/pandas/base.py:252: error: Incompatible return value type (got "_Wrapped[Callable[..., None], Column, Callable[[Union[Series[Any], Index], VarArg(Any)], Union[Series[Any], Index]]]", expected "Callable[..., Union[Series[Any], Index]]")  [return-value]
+ python/pyspark/pandas/base.py:252: note: "_Wrapped[Callable[..., None], Column, Callable[[Union[Series[Any], Index], VarArg(Any)], Union[Series[Any], Index]]].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Column]"
+ python/pyspark/pandas/base.py:269: error: Incompatible return value type (got "_Wrapped[Callable[..., None], Column, Callable[[Union[Series[Any], Index], VarArg(Any)], Union[Series[Any], Index]]]", expected "Callable[..., Union[Series[Any], Index]]")  [return-value]
+ python/pyspark/pandas/base.py:269: note: "_Wrapped[Callable[..., None], Column, Callable[[Union[Series[Any], Index], VarArg(Any)], Union[Series[Any], Index]]].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Column]"

rotki (https://github.com/rotki/rotki)
+ rotkehlchen/api/v1/parser.py:67: error: Unused "type: ignore" comment

tornado (https://github.com/tornadoweb/tornado)
+ tornado/gen.py:270: error: Incompatible return value type (got "_Wrapped[Callable[..., None], object, Callable[[VarArg(Any), KwArg(Any)], Future[_T]]]", expected "Callable[..., Future[_T]]")
+ tornado/gen.py:270: note: "_Wrapped[Callable[..., None], object, Callable[[VarArg(Any), KwArg(Any)], Future[_T]]].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], object]"
- tornado/testing.py:595: error: Incompatible types in assignment (expression has type "Callable[..., Future[Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]]", variable has type "Callable[[AsyncTestCase, VarArg(Any), KwArg(Any)], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]")
+ tornado/testing.py:595: error: Incompatible types in assignment (expression has type "Callable[..., Future[Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]]", variable has type "_Wrapped[Callable[..., None], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]], Callable[[AsyncTestCase, VarArg(Any), KwArg(Any)], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]]")
+ tornado/testing.py:621: error: Incompatible return value type (got "_Wrapped[Callable[[VarArg(Any), KwArg(Any)], None], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]], Callable[[AsyncTestCase, VarArg(Any), KwArg(Any)], None]]", expected "Callable[..., None]")
+ tornado/testing.py:621: note: "_Wrapped[Callable[[VarArg(Any), KwArg(Any)], None], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]], Callable[[AsyncTestCase, VarArg(Any), KwArg(Any)], None]].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]"

pip (https://github.com/pypa/pip)
+ src/pip/_internal/index/collector.py:293: error: Incompatible return value type (got "_Wrapped[Callable[[HTMLPage], None], Iterable[Link], Callable[[HTMLPage], List[Link]]]", expected "Callable[[HTMLPage], List[Link]]")
+ src/pip/_internal/index/collector.py:293: note: "_Wrapped[Callable[[HTMLPage], None], Iterable[Link], Callable[[HTMLPage], List[Link]]].__call__" has type "Callable[[HTMLPage], Iterable[Link]]"

boostedblob (https://github.com/hauntsaninja/boostedblob)
+ boostedblob/cli.py:30: error: Incompatible return value type (got "_Wrapped[Callable[..., None], Awaitable[T], Callable[[VarArg(Any), KwArg(Any)], Any]]", expected "Callable[..., T]")
+ boostedblob/cli.py:30: note: "_Wrapped[Callable[..., None], Awaitable[T], Callable[[VarArg(Any), KwArg(Any)], Any]].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Awaitable[T]]"

kopf (https://github.com/nolar/kopf)
+ kopf/_kits/webhacks.py:80: error: Argument 1 to "wraps" has incompatible type "_ServerFn"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/formparser.py:153: error: Incompatible return value type (got "_Wrapped[Any, Any, Any]", expected "F")  [return-value]

bokeh (https://github.com/bokeh/bokeh)
+ bokeh/document/callbacks.py: note: In member "add_session_callback" of class "DocumentCallbackManager":
+ bokeh/document/callbacks.py:182:31: error: Incompatible types in assignment (expression has type "Callable[[], None]", variable has type "_Wrapped[Any, Any, Any]")  [assignment]

pandas (https://github.com/pandas-dev/pandas)
+ pandas/util/_decorators.py:90: error: Unused "type: ignore" comment
+ pandas/core/strings/accessor.py:127: error: "_Wrapped[Any, Any, Any]" has no attribute "__name__"  [attr-defined]

ibis (https://github.com/ibis-project/ibis)
- ibis/backends/impala/hdfs.py:464: error: Return type "Dict[Any, Any]" of "exists" incompatible with return type "bool" in supertype "HDFS"
- ibis/backends/impala/hdfs.py:615: error: Signature of "get" incompatible with supertype "HDFS"
- ibis/backends/impala/hdfs.py:615: note:      Superclass:
- ibis/backends/impala/hdfs.py:615: note:          def get(self, hdfs_path: str, local_path: str = ..., overwrite: bool = ...) -> str
- ibis/backends/impala/hdfs.py:615: note:      Subclass:
- ibis/backends/impala/hdfs.py:615: note:          def get(self, hdfs_path: str, local_path: str, overwrite: bool = ..., verbose: Optional[bool] = ..., **kwargs: Any) -> str

tornado (https://github.com/tornadoweb/tornado)
- tornado/testing.py:595: error: Incompatible types in assignment (expression has type "Callable[..., Future[Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]]", variable has type "Callable[[AsyncTestCase, VarArg(Any), KwArg(Any)], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]")
+ tornado/testing.py:595: error: Incompatible types in assignment (expression has type "Callable[..., Future[Any]]", variable has type "_Wrapped[Any, Any, Any]")

spark (https://github.com/apache/spark)
+ python/pyspark/sql/udf.py:278: error: "_Wrapped[Any, Any, Any]" has no attribute "__name__"  [attr-defined]

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/fixtures.py:1134: error: Argument 1 to "wraps" has incompatible type "FixtureFunction"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]

zulip (https://github.com/zulip/zulip)
+ zerver/lib/type_debug.py:75: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/profile.py:27: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/cache.py:143: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/cache.py:176: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/openapi/python_examples.py:44: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ corporate/lib/stripe.py:247: error: Argument 1 to "wraps" has incompatible type "CallableT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/decorator.py:871: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/tests/test_service_bot_system.py:419: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/formparser.py:153: error: Incompatible return value type (got "_Wrapped[<nothing>, <nothing>, <nothing>, <nothing>]", expected "F")  [return-value]

bokeh (https://github.com/bokeh/bokeh)
+ bokeh/document/callbacks.py: note: In member "add_session_callback" of class "DocumentCallbackManager":
+ bokeh/document/callbacks.py:182:31: error: Incompatible types in assignment (expression has type "Callable[[], None]", variable has type "_Wrapped[Callable[[], None], None, Callable[[], None], None]")  [assignment]

pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/strings/accessor.py:127: error: "_Wrapped[<nothing>, <nothing>, Callable[[Any, VarArg(Any), KwArg(Any)], None], Any]" has no attribute "__name__"  [attr-defined]

ibis (https://github.com/ibis-project/ibis)
- ibis/backends/impala/hdfs.py:464: error: Return type "Dict[Any, Any]" of "exists" incompatible with return type "bool" in supertype "HDFS"
+ ibis/backends/impala/hdfs.py:464: error: Signature of "exists" incompatible with supertype "HDFS"
+ ibis/backends/impala/hdfs.py:529: error: Missing positional argument "path" in call to "__call__" of "_Wrapped"
+ ibis/backends/impala/hdfs.py:529: error: Argument 1 to "__call__" of "_Wrapped" has incompatible type "str"; expected "HDFS"
- ibis/backends/impala/hdfs.py:615: error: Signature of "get" incompatible with supertype "HDFS"
- ibis/backends/impala/hdfs.py:615: note:      Superclass:
- ibis/backends/impala/hdfs.py:615: note:          def get(self, hdfs_path: str, local_path: str = ..., overwrite: bool = ...) -> str
- ibis/backends/impala/hdfs.py:615: note:      Subclass:
- ibis/backends/impala/hdfs.py:615: note:          def get(self, hdfs_path: str, local_path: str, overwrite: bool = ..., verbose: Optional[bool] = ..., **kwargs: Any) -> str

tornado (https://github.com/tornadoweb/tornado)
- tornado/testing.py:595: error: Incompatible types in assignment (expression has type "Callable[..., Future[Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]]", variable has type "Callable[[AsyncTestCase, VarArg(Any), KwArg(Any)], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]")
+ tornado/testing.py:595: error: Incompatible types in assignment (expression has type "Callable[..., Future[Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]]", variable has type "_Wrapped[Callable[..., None], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]], Callable[[AsyncTestCase, VarArg(Any), KwArg(Any)], None], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]")

spark (https://github.com/apache/spark)
+ python/pyspark/sql/udf.py:278: error: "_Wrapped[Callable[..., None], Any, Callable[[VarArg(Union[Column, str])], None], Column]" has no attribute "__name__"  [attr-defined]

core (https://github.com/home-assistant/core)
+ homeassistant/components/mqtt/__init__.py:378: error: Incompatible types in assignment (expression has type "_Wrapped[Callable[[str, Union[str, bytes], int], None], Optional[Awaitable[None]], Callable[[ReceiveMessage], None], Coroutine[Any, Any, None]]", variable has type "Union[Callable[[ReceiveMessage], Awaitable[None]], Callable[[ReceiveMessage], None]]")  [assignment]
+ homeassistant/components/mqtt/__init__.py:386: error: Incompatible types in assignment (expression has type "_Wrapped[Callable[[str, Union[str, bytes], int], None], Optional[Awaitable[None]], Callable[[ReceiveMessage], None], None]", variable has type "Union[Callable[[ReceiveMessage], Awaitable[None]], Callable[[ReceiveMessage], None]]")  [assignment]

kopf (https://github.com/nolar/kopf)
+ kopf/_kits/webhacks.py:80: error: Argument 1 to "wraps" has incompatible type "_ServerFn"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/fixtures.py:1134: error: Argument 1 to "wraps" has incompatible type "FixtureFunction"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]

zulip (https://github.com/zulip/zulip)
+ zerver/lib/type_debug.py:75: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/profile.py:27: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/cache.py:143: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/cache.py:176: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/openapi/python_examples.py:44: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ corporate/lib/stripe.py:247: error: Argument 1 to "wraps" has incompatible type "CallableT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/decorator.py:871: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/tests/test_service_bot_system.py:419: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]

pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/strings/accessor.py:127: error: "_Wrapped[<nothing>, <nothing>, Callable[[Any, VarArg(Any), KwArg(Any)], None], Any]" has no attribute "__name__"  [attr-defined]

bokeh (https://github.com/bokeh/bokeh)
+ bokeh/document/callbacks.py: note: In member "add_session_callback" of class "DocumentCallbackManager":
+ bokeh/document/callbacks.py:182:31: error: Incompatible types in assignment (expression has type "Callable[[], None]", variable has type "_Wrapped[Callable[[], None], None, Callable[[], None], None]")  [assignment]

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/formparser.py:153: error: Incompatible return value type (got "_Wrapped[<nothing>, <nothing>, <nothing>, <nothing>]", expected "F")  [return-value]

ibis (https://github.com/ibis-project/ibis)
- ibis/backends/impala/hdfs.py:464: error: Return type "Dict[Any, Any]" of "exists" incompatible with return type "bool" in supertype "HDFS"
+ ibis/backends/impala/hdfs.py:464: error: Signature of "exists" incompatible with supertype "HDFS"
+ ibis/backends/impala/hdfs.py:529: error: Missing positional argument "path" in call to "__call__" of "_Wrapped"
+ ibis/backends/impala/hdfs.py:529: error: Argument 1 to "__call__" of "_Wrapped" has incompatible type "str"; expected "HDFS"
- ibis/backends/impala/hdfs.py:615: error: Signature of "get" incompatible with supertype "HDFS"
- ibis/backends/impala/hdfs.py:615: note:      Superclass:
- ibis/backends/impala/hdfs.py:615: note:          def get(self, hdfs_path: str, local_path: str = ..., overwrite: bool = ...) -> str
- ibis/backends/impala/hdfs.py:615: note:      Subclass:
- ibis/backends/impala/hdfs.py:615: note:          def get(self, hdfs_path: str, local_path: str, overwrite: bool = ..., verbose: Optional[bool] = ..., **kwargs: Any) -> str

tornado (https://github.com/tornadoweb/tornado)
- tornado/testing.py:595: error: Incompatible types in assignment (expression has type "Callable[..., Future[Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]]", variable has type "Callable[[AsyncTestCase, VarArg(Any), KwArg(Any)], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]")
+ tornado/testing.py:595: error: Incompatible types in assignment (expression has type "Callable[..., Future[Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]]", variable has type "_Wrapped[Callable[..., None], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]], Callable[[AsyncTestCase, VarArg(Any), KwArg(Any)], None], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]")

spark (https://github.com/apache/spark)
+ python/pyspark/sql/udf.py:278: error: "_Wrapped[Callable[..., None], Any, Callable[[VarArg(Union[Column, str])], None], Column]" has no attribute "__name__"  [attr-defined]

core (https://github.com/home-assistant/core)
+ homeassistant/components/mqtt/__init__.py:378: error: Incompatible types in assignment (expression has type "_Wrapped[Callable[[str, Union[str, bytes], int], None], Optional[Awaitable[None]], Callable[[ReceiveMessage], None], Coroutine[Any, Any, None]]", variable has type "Union[Callable[[ReceiveMessage], Awaitable[None]], Callable[[ReceiveMessage], None]]")  [assignment]
+ homeassistant/components/mqtt/__init__.py:386: error: Incompatible types in assignment (expression has type "_Wrapped[Callable[[str, Union[str, bytes], int], None], Optional[Awaitable[None]], Callable[[ReceiveMessage], None], None]", variable has type "Union[Callable[[ReceiveMessage], Awaitable[None]], Callable[[ReceiveMessage], None]]")  [assignment]

kopf (https://github.com/nolar/kopf)
+ kopf/_kits/webhacks.py:80: error: Argument 1 to "wraps" has incompatible type "_ServerFn"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"

[srittau]

The PR certainly seems worthwhile and correct to me, but we still need pytype and mypy fixes for it to be viable. I suggest we keep it open as "deferred" for now.

[rchen152]

The pytype issue in this PR has been fixed in the latest release (pytype-2022.1.5).

[erictraut]

Thanks @rchen152! Looks like we'll need to update the pytype CI tests in typeshed to use the new version.

@srittau, do you have any remaining concerns about the PR?

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

zulip (https://github.com/zulip/zulip)
+ zerver/lib/type_debug.py:75: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/profile.py:27: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/cache.py:143: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/cache.py:176: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/openapi/python_examples.py:44: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ corporate/lib/stripe.py:247: error: Argument 1 to "wraps" has incompatible type "CallableT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/decorator.py:875: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/tests/test_service_bot_system.py:419: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]

pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/fixtures.py:1134: error: Argument 1 to "wraps" has incompatible type "FixtureFunction"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]

bokeh (https://github.com/bokeh/bokeh)
+ bokeh/document/callbacks.py: note: In member "add_session_callback" of class "DocumentCallbackManager":
+ bokeh/document/callbacks.py:182:31: error: Incompatible types in assignment (expression has type "Callable[[], None]", variable has type "_Wrapped[Callable[[], None], None, Callable[[], None], None]")  [assignment]

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/formparser.py:153: error: Incompatible return value type (got "_Wrapped[<nothing>, <nothing>, <nothing>, <nothing>]", expected "F")  [return-value]

pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/strings/accessor.py:127: error: "_Wrapped[<nothing>, <nothing>, Callable[[Any, VarArg(Any), KwArg(Any)], None], Any]" has no attribute "__name__"  [attr-defined]

ibis (https://github.com/ibis-project/ibis)
- ibis/backends/impala/hdfs.py:464: error: Return type "Dict[Any, Any]" of "exists" incompatible with return type "bool" in supertype "HDFS"
+ ibis/backends/impala/hdfs.py:464: error: Signature of "exists" incompatible with supertype "HDFS"
+ ibis/backends/impala/hdfs.py:529: error: Missing positional argument "path" in call to "__call__" of "_Wrapped"
+ ibis/backends/impala/hdfs.py:529: error: Argument 1 to "__call__" of "_Wrapped" has incompatible type "str"; expected "HDFS"
- ibis/backends/impala/hdfs.py:615: error: Signature of "get" incompatible with supertype "HDFS"
- ibis/backends/impala/hdfs.py:615: note:      Superclass:
- ibis/backends/impala/hdfs.py:615: note:          def get(self, hdfs_path: str, local_path: str = ..., overwrite: bool = ...) -> str
- ibis/backends/impala/hdfs.py:615: note:      Subclass:
- ibis/backends/impala/hdfs.py:615: note:          def get(self, hdfs_path: str, local_path: str, overwrite: bool = ..., verbose: Optional[bool] = ..., **kwargs: Any) -> str

tornado (https://github.com/tornadoweb/tornado)
- tornado/testing.py:595: error: Incompatible types in assignment (expression has type "Callable[..., Future[Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]]", variable has type "Callable[[AsyncTestCase, VarArg(Any), KwArg(Any)], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]")
+ tornado/testing.py:595: error: Incompatible types in assignment (expression has type "Callable[..., Future[Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]]", variable has type "_Wrapped[Callable[..., None], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]], Callable[[AsyncTestCase, VarArg(Any), KwArg(Any)], None], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]")

spark (https://github.com/apache/spark)
+ python/pyspark/sql/udf.py:278: error: "_Wrapped[Callable[..., None], Any, Callable[[VarArg(Union[Column, str])], None], Column]" has no attribute "__name__"  [attr-defined]

core (https://github.com/home-assistant/core)
+ homeassistant/components/mqtt/__init__.py:489: error: Incompatible types in assignment (expression has type "_Wrapped[Callable[[str, Union[str, bytes], int], None], Optional[Awaitable[None]], Callable[[ReceiveMessage], None], Coroutine[Any, Any, None]]", variable has type "Union[Callable[[ReceiveMessage], Awaitable[None]], Callable[[ReceiveMessage], None]]")  [assignment]
+ homeassistant/components/mqtt/__init__.py:497: error: Incompatible types in assignment (expression has type "_Wrapped[Callable[[str, Union[str, bytes], int], None], Optional[Awaitable[None]], Callable[[ReceiveMessage], None], None]", variable has type "Union[Callable[[ReceiveMessage], Awaitable[None]], Callable[[ReceiveMessage], None]]")  [assignment]

kopf (https://github.com/nolar/kopf)
+ kopf/_kits/webhacks.py:80: error: Argument 1 to "wraps" has incompatible type "_ServerFn"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"

[AlexWaygood]

The spark entry in the primer diff concerns me. It's going to be far more common to use @wraps with functions than with any other kind of callable, and it seems that, with this patch, mypy no longer recognises that a function will still have a __name__ attribute even if it's decorated with @wraps. I'm guessing that also applies for __defaults__, __kwdefaults__, and all the other special attributes that exist for functions but not for other callables.

[srittau]

The primer output is still a bit concerning, but I think mostly unavoidable with this PR. Any other opinions on that? I am removing the "deferred" label for now.

[AlexWaygood]

The spark entry in the primer diff concerns me.

I have an idea for how we could get rid of this false-positive (and the similar one for pandas), but:

• My idea is a little complicated.
• I'm not sure my idea will work.
• I can always try it out in a separate PR after this one is merged, to see what mypy_primer thinks.

So, please don't block this PR solely based on my concerns :)

[erictraut]

Any other thoughts on this PR?

Incidentally, the proposed change addresses one of the most upvoted issues for pylance, which is why I'd really like to see this fix (or something similar) make it into typeshed.

[JelleZijlstra]

Let's do it, though I'm still worried this will create false positives for places that use wraps() but change the signature.

[JelleZijlstra]

(And I hope I fixed the merge conflict right this time)

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

zulip (https://github.com/zulip/zulip)
+ zerver/lib/type_debug.py:75: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/profile.py:27: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/cache.py:143: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/lib/cache.py:176: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/openapi/python_examples.py:44: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ corporate/lib/stripe.py:247: error: Argument 1 to "wraps" has incompatible type "CallableT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/decorator.py:879: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/tests/test_service_bot_system.py:419: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]

pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/fixtures.py:1134: error: Argument 1 to "wraps" has incompatible type "FixtureFunction"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]

bokeh (https://github.com/bokeh/bokeh)
+ bokeh/document/callbacks.py: note: In member "add_session_callback" of class "DocumentCallbackManager":
+ bokeh/document/callbacks.py:182:31: error: Incompatible types in assignment (expression has type "Callable[[], None]", variable has type "_Wrapped[Callable[[], None], None, Callable[[], None], None]")  [assignment]

pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/strings/accessor.py:127: error: "_Wrapped[<nothing>, <nothing>, Callable[[Any, VarArg(Any), KwArg(Any)], None], Any]" has no attribute "__name__"  [attr-defined]

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/formparser.py:153: error: Incompatible return value type (got "_Wrapped[<nothing>, <nothing>, <nothing>, <nothing>]", expected "F")  [return-value]

tornado (https://github.com/tornadoweb/tornado)
- tornado/testing.py:651: error: Incompatible types in assignment (expression has type "Callable[..., Future[Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]]", variable has type "Callable[[AsyncTestCase, VarArg(Any), KwArg(Any)], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]")
+ tornado/testing.py:651: error: Incompatible types in assignment (expression has type "Callable[..., Future[Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]]", variable has type "_Wrapped[Callable[..., None], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]], Callable[[AsyncTestCase, VarArg(Any), KwArg(Any)], None], Union[Generator[Any, Any, Any], Coroutine[Any, Any, Any]]]")

ibis (https://github.com/ibis-project/ibis)
- ibis/backends/impala/hdfs.py:464: error: Return type "Dict[Any, Any]" of "exists" incompatible with return type "bool" in supertype "HDFS"
+ ibis/backends/impala/hdfs.py:464: error: Signature of "exists" incompatible with supertype "HDFS"
+ ibis/backends/impala/hdfs.py:529: error: Missing positional argument "path" in call to "__call__" of "_Wrapped"
+ ibis/backends/impala/hdfs.py:529: error: Argument 1 to "__call__" of "_Wrapped" has incompatible type "str"; expected "HDFS"
- ibis/backends/impala/hdfs.py:615: error: Signature of "get" incompatible with supertype "HDFS"
- ibis/backends/impala/hdfs.py:615: note:      Superclass:
- ibis/backends/impala/hdfs.py:615: note:          def get(self, hdfs_path: str, local_path: str = ..., overwrite: bool = ...) -> str
- ibis/backends/impala/hdfs.py:615: note:      Subclass:
- ibis/backends/impala/hdfs.py:615: note:          def get(self, hdfs_path: str, local_path: str, overwrite: bool = ..., verbose: Optional[bool] = ..., **kwargs: Any) -> str

core (https://github.com/home-assistant/core)
+ homeassistant/components/mqtt/__init__.py:480: error: Incompatible types in assignment (expression has type "_Wrapped[Callable[[str, Union[str, bytes], int], None], Optional[Awaitable[None]], Callable[[ReceiveMessage], None], Coroutine[Any, Any, None]]", variable has type "Union[Callable[[ReceiveMessage], Awaitable[None]], Callable[[ReceiveMessage], None]]")  [assignment]
+ homeassistant/components/mqtt/__init__.py:488: error: Incompatible types in assignment (expression has type "_Wrapped[Callable[[str, Union[str, bytes], int], None], Optional[Awaitable[None]], Callable[[ReceiveMessage], None], None]", variable has type "Union[Callable[[ReceiveMessage], Awaitable[None]], Callable[[ReceiveMessage], None]]")  [assignment]

spark (https://github.com/apache/spark)
+ python/pyspark/sql/udf.py:278: error: "_Wrapped[Callable[..., None], Any, Callable[[VarArg(Union[Column, str])], None], Column]" has no attribute "__name__"  [attr-defined]

kopf (https://github.com/nolar/kopf)
+ kopf/_kits/webhacks.py:80: error: Argument 1 to "wraps" has incompatible type "_ServerFn"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"

[Akuli]

false positives for places that use wraps() but change the signature

These are arguably true positives, as one reason to use wraps() is that it copies the signature. A type checker should warn about this IMO.

[erictraut]

Is this PR ready to merge, or are we still waiting for additional input?

[JelleZijlstra]

The mypy-primer results indicate this change may be disruptive for mypy. I haven't looked at the code in detail yet.

[DevilXD]

@erictraut Is usage like this supposed to give errors?

AnyFunc = TypeVar("AnyFunc", bound=Callable[..., object])

def deco(func: AnyFunc) -> AnyFunc:
    @functools.wraps(func)
    def wrapper(*args, **kwargs) -> object:
        ...
        return func(*args, **kwargs)
    return wrapper

Argument 1 to "wraps" has incompatible type "AnyFunc"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]

According to the primer results, this error type pops up in zulip 8 times, once in pytest and once in kopf. I'm not 100% sure, but the usage above looks valid to me. If that's not the case, please correct me.

EDIT: Actually, it may be due to: https://github.com/python/mypy/issues/9075

[erictraut]

I think it's correct for this to be flagged as an error. The signature of the function being wrapped (which is the input parameter func) is not the necessarily the same as the signature of the resulting wrapped function, so they cannot be represented by the same type variable. This would probably require a cast(AnyFunc, wrapper).

Both pyright and mypy flag this as an error today, prior to the proposed change in typeshed's functools.pyi.

[DevilXD]

I see. What about this?

ibis (https://github.com/ibis-project/ibis)
+ ibis/backends/impala/hdfs.py:464: error: Signature of "exists" incompatible with supertype "HDFS"
+ ibis/backends/impala/hdfs.py:529: error: Missing positional argument "path" in call to "__call__" of "_Wrapped"
+ ibis/backends/impala/hdfs.py:529: error: Argument 1 to "__call__" of "_Wrapped" has incompatible type "str"; expected "HDFS"

The code appears to be roughly:

from functools import wraps as implements

class HDFS:
    def exists(self, path: str) -> bool:
        ...

class WebHDFS(HDFS):
    @implements(HDFS.exists)
    def exists(self, path: str) -> dict:  # L464
        ...

    def mv(self) -> None:
        self.exists("...")  # L529

Ignoring the existing dict over bool incompatible signature error, it seems that the self argument is somewhat considered in the resulting call, meaning that "..." becomes self and since there's no argument for path, MyPy complains about it.

Other than that, and the error where "_Wrapped has no attribute __name__", which is kinda expected to be a thing here (could be remedied by adding it to _Wrapped as __name__: str though), it's pretty much it when it come to addressing the primer results.

[erictraut]

Yeah, that case is problematic because of the way ParamSpec captures the self as part of the signature. I don't have a good suggestion for how to work around problem. Here is a discussion that touches on this issue and asks the authors of PEP 612 for guidance. Unfortunately, I don't think the suggestion they provide in that discussion (which is about @functools.cache) works for @functools.wraps.

Unless someone has a better suggestion, we'll need to decide whether a false positive in this case (where the argument passed to wraps() is an instance method) is common enough to outweigh the benefits that this change otherwise provides. I'll just reiterate that the current handling of wraps generates a lot of complaints from pyright and pylance users, so I'm motivated to find some fix.

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/fixtures.py:1150: error: Argument 1 to "wraps" has incompatible type "FixtureFunction"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]

zulip (https://github.com/zulip/zulip)
+ zerver/lib/type_debug.py:75: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/openapi/python_examples.py:44: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
+ zerver/tests/test_service_bot_system.py:420: error: Argument 1 to "wraps" has incompatible type "FuncT"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]

pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/strings/accessor.py:127: error: "_Wrapped[<nothing>, <nothing>, Callable[[Any, VarArg(Any), KwArg(Any)], None], Any]" has no attribute "__name__"  [attr-defined]

bokeh (https://github.com/bokeh/bokeh)
+ bokeh/document/callbacks.py: note: In member "add_session_callback" of class "DocumentCallbackManager":
+ bokeh/document/callbacks.py:181:31: error: Incompatible types in assignment (expression has type "Callable[[], None]", variable has type "_Wrapped[Callable[[], None], None, Callable[[], None], None]")  [assignment]

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/formparser.py:153: error: Incompatible return value type (got "_Wrapped[<nothing>, <nothing>, <nothing>, <nothing>]", expected "F")  [return-value]
- tests/test_wrappers.py:1084: note: Use "-> None" if function does not return a value
- tests/test_wrappers.py:1109: note: (Skipping most remaining errors due to unresolved imports or missing stubs; fix these first)
+ tests/test_wrappers.py:1084: note: (Skipping most remaining errors due to unresolved imports or missing stubs; fix these first)

core (https://github.com/home-assistant/core)
+ homeassistant/components/mqtt/__init__.py:511: error: Incompatible types in assignment (expression has type "_Wrapped[Callable[[str, Union[str, bytes], int], None], Optional[Awaitable[None]], Callable[[ReceiveMessage], None], Coroutine[Any, Any, None]]", variable has type "Union[Callable[[ReceiveMessage], Awaitable[None]], Callable[[ReceiveMessage], None]]")  [assignment]
+ homeassistant/components/mqtt/__init__.py:519: error: Incompatible types in assignment (expression has type "_Wrapped[Callable[[str, Union[str, bytes], int], None], Optional[Awaitable[None]], Callable[[ReceiveMessage], None], None]", variable has type "Union[Callable[[ReceiveMessage], Awaitable[None]], Callable[[ReceiveMessage], None]]")  [assignment]

kopf (https://github.com/nolar/kopf)
+ kopf/_kits/webhacks.py:80: error: Argument 1 to "wraps" has incompatible type "_ServerFn"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"

[erictraut]

Are there any updates on this PR? I'm unclear on whether there's anything else I need to do to get it merged.

We continue to get bug reports from pyright users about decorators that use wraps. Without this fix, I'm not sure what to tell them.