Revert "Revert use of `ParamSpec` for `functools.wraps`"

[tamird]

This is an experiment to see if we can remove https://github.com/python/mypy/blob/790e8a73d8671a41cae419b4ea07579bfb2bc292/misc/sync-typeshed.py#L185 and https://github.com/python/mypy/blob/790e8a73d8671a41cae419b4ea07579bfb2bc292/misc/generate_changelog.py#L82.

/cc @AlexWaygood

[jakkdl]

This is an attempt to fix e.g. https://github.com/python/typeshed/issues/10653

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

jinja (https://github.com/pallets/jinja)
+ src/jinja2/compiler.py:56: error: Incompatible return value type (got "_Wrapped[[VarArg(Any), KwArg(Any)], Any, [VarArg(Any), KwArg(Any)], Any]", expected "F")  [return-value]

prefect (https://github.com/PrefectHQ/prefect)
- src/prefect/utilities/asyncutils.py:222: error: Argument 1 to "wraps" has incompatible type "T"; expected "Callable[..., object]"  [arg-type]
+ src/prefect/utilities/asyncutils.py:222: error: Argument 1 to "wraps" has incompatible type "T"; expected "Callable[[VarArg(Never), KwArg(Never)], Never]"  [arg-type]
+ src/prefect/utilities/asyncutils.py:264: error: Need type annotation for "wrapper"  [var-annotated]
- src/prefect/utilities/asyncutils.py:270: error: "Callable[[VarArg(Any), KwArg(Any)], Any]" has no attribute "aio"  [attr-defined]
+ src/prefect/utilities/asyncutils.py:270: error: "_Wrapped[Any, Any, [VarArg(Any), KwArg(Any)], Any]" has no attribute "aio"  [attr-defined]
- src/prefect/utilities/asyncutils.py:271: error: Incompatible return value type (got "Callable[[VarArg(Any), KwArg(Any)], Any]", expected "T")  [return-value]
+ src/prefect/utilities/asyncutils.py:271: error: Incompatible return value type (got "_Wrapped[Any, Any, [VarArg(Any), KwArg(Any)], Any]", expected "T")  [return-value]

nox (https://github.com/wntrblm/nox)
+ nox/_decorators.py:54: error: Incompatible types in assignment (expression has type "_Wrapped[[VarArg(Any), KwArg(Any)], Any, [VarArg(Any), KwArg(Any)], Any]", variable has type "FunctionType")  [assignment]

tornado (https://github.com/tornadoweb/tornado)
+ tornado/gen.py:268: error: Unused "type: ignore" comment  [unused-ignore]

trio (https://github.com/python-trio/trio)
+ src/trio/_core/_ki.py:142: error: Type of decorated function contains type "Any" ("_Wrapped[ArgsT, CoroutineType[Any, Any, Any], ArgsT, RetT]")  [misc]
+ src/trio/_core/_ki.py:152: error: Type of decorated function contains type "Any" ("_Wrapped[ArgsT, GeneratorType[Any, Any, Any], ArgsT, RetT]")  [misc]
+ src/trio/_core/_ki.py:168: error: Argument 1 to "wraps" has incompatible type "Union[Callable[ArgsT, AsyncGeneratorType[Any, Any]], Callable[..., AsyncGeneratorType[object, object]]]"; expected "Callable[[VarArg(object), KwArg(object)], AsyncGeneratorType[Any, Any]]"  [arg-type]
+ src/trio/_core/_ki.py:169: error: Type of decorated function contains type "Any" ("_Wrapped[object, AsyncGeneratorType[Any, Any], ArgsT, RetT]")  [misc]

jax (https://github.com/google/jax)
+ jax/_src/maps.py:627: error: Incompatible return value type (got "_Wrapped[[VarArg(Any), KwArg(Any)], Any, [VarArg(Any), KwArg(Any)], Any]", expected "Wrapped")  [return-value]
+ jax/_src/maps.py:627: note: "_Wrapped" is missing following "Wrapped" protocol member:
+ jax/_src/maps.py:627: note:     lower

bokeh (https://github.com/bokeh/bokeh)
+ src/bokeh/document/callbacks.py: note: In member "add_session_callback" of class "DocumentCallbackManager":
+ src/bokeh/document/callbacks.py:185:31: error: Incompatible types in assignment (expression has type "Callable[[], None]", variable has type "_Wrapped[[], None, [], None]")  [assignment]

ibis (https://github.com/ibis-project/ibis)
+ ibis/expr/types/relations.py:3035: error: Missing positional argument "predicates" in call to "__call__" of "_Wrapped"  [call-arg]

hydra-zen (https://github.com/mit-ll-responsible-ai/hydra-zen)
- src/hydra_zen/wrapper/_implementations.py:503: error: Incompatible types in assignment (expression has type "Zen[P, R]", variable has type "Callable[[Any], Any]")  [assignment]
+ src/hydra_zen/wrapper/_implementations.py:503: error: Incompatible types in assignment (expression has type "Zen[P, R]", variable has type "_Wrapped[P, R, [Any], Any]")  [assignment]
- src/hydra_zen/wrapper/_implementations.py:503: note: "Zen[P, R].__call__" has type "Callable[[DataClass_ | type[DataClass_] | dict[Any, Any] | DictConfig | str], R]"

[AlexWaygood]

Primer summary:

• 10 new errors in user code
• 1 existing error goes away
• 4 existing errors have their error message change slightly

The next step is to go through the new errors and analyse whether they're true positives or false positives

[jakkdl]

The ones in trio are true positives afaict @TeamSpen210

[TeamSpen210]

Yeah, that block of code's not particularly typeable, the problem is inspect's functions causing typevars to be discarded. It might be fixable with TypeIs, actually. In any case it's not a problem for this PR.

[tamird]

jinja: looks like they're already using cast, so doesn't seem like a meaningful regression. https://github.com/pallets/jinja/blob/3fd91e4d11bdd131d8c12805177dbe74d85e7b82/src/jinja2/compiler.py#L56

nox: ditto, cast is used a few lines down. https://github.com/wntrblm/nox/blob/93bacbda291763171813f12cd3d0db97d852439e/nox/_decorators.py#L54-L56

jax: looks like a true positive? https://github.com/google/jax/blob/c268df91b70572803e5c54bec530c6d7e3889e99/jax/_src/maps.py#L626

bokeh: annotated function type != unannotated function type. false positive? https://github.com/bokeh/bokeh/blob/17b0187a7efac45b27be081aa80416938da3f899/src/bokeh/document/callbacks.py#L183-L185

ibis: false positive? superficially looks like predicates is passed https://github.com/ibis-project/ibis/blob/7667328ada9cb8ef6bb4c456636c6a7ccab2ad1a/ibis/expr/types/relations.py#L3035-L3037

[tamird]

What's next here?

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

jinja (https://github.com/pallets/jinja)
+ src/jinja2/compiler.py:56: error: Incompatible return value type (got "_Wrapped[[VarArg(Any), KwArg(Any)], Any, [VarArg(Any), KwArg(Any)], Any]", expected "F")  [return-value]

prefect (https://github.com/PrefectHQ/prefect)
- src/prefect/utilities/asyncutils.py:222: error: Argument 1 to "wraps" has incompatible type "T"; expected "Callable[..., object]"  [arg-type]
+ src/prefect/utilities/asyncutils.py:222: error: Argument 1 to "wraps" has incompatible type "T"; expected "Callable[[VarArg(Never), KwArg(Never)], Never]"  [arg-type]
+ src/prefect/utilities/asyncutils.py:264: error: Need type annotation for "wrapper"  [var-annotated]
- src/prefect/utilities/asyncutils.py:270: error: "Callable[[VarArg(Any), KwArg(Any)], Any]" has no attribute "aio"  [attr-defined]
+ src/prefect/utilities/asyncutils.py:270: error: "_Wrapped[Any, Any, [VarArg(Any), KwArg(Any)], Any]" has no attribute "aio"  [attr-defined]
- src/prefect/utilities/asyncutils.py:271: error: Incompatible return value type (got "Callable[[VarArg(Any), KwArg(Any)], Any]", expected "T")  [return-value]
+ src/prefect/utilities/asyncutils.py:271: error: Incompatible return value type (got "_Wrapped[Any, Any, [VarArg(Any), KwArg(Any)], Any]", expected "T")  [return-value]

tornado (https://github.com/tornadoweb/tornado)
+ tornado/gen.py:268: error: Unused "type: ignore" comment  [unused-ignore]

nox (https://github.com/wntrblm/nox)
+ nox/_decorators.py:54: error: Incompatible types in assignment (expression has type "_Wrapped[[VarArg(Any), KwArg(Any)], Any, [VarArg(Any), KwArg(Any)], Any]", variable has type "FunctionType")  [assignment]

trio (https://github.com/python-trio/trio)
+ src/trio/_core/_ki.py:142: error: Type of decorated function contains type "Any" ("_Wrapped[ArgsT, CoroutineType[Any, Any, Any], ArgsT, RetT]")  [misc]
+ src/trio/_core/_ki.py:152: error: Type of decorated function contains type "Any" ("_Wrapped[ArgsT, GeneratorType[Any, Any, Any], ArgsT, RetT]")  [misc]
+ src/trio/_core/_ki.py:168: error: Argument 1 to "wraps" has incompatible type "Union[Callable[ArgsT, AsyncGeneratorType[Any, Any]], Callable[..., AsyncGeneratorType[object, object]]]"; expected "Callable[[VarArg(object), KwArg(object)], AsyncGeneratorType[Any, Any]]"  [arg-type]
+ src/trio/_core/_ki.py:169: error: Type of decorated function contains type "Any" ("_Wrapped[object, AsyncGeneratorType[Any, Any], ArgsT, RetT]")  [misc]

jax (https://github.com/google/jax)
+ jax/_src/maps.py:627: error: Incompatible return value type (got "_Wrapped[[VarArg(Any), KwArg(Any)], Any, [VarArg(Any), KwArg(Any)], Any]", expected "Wrapped")  [return-value]
+ jax/_src/maps.py:627: note: "_Wrapped" is missing following "Wrapped" protocol member:
+ jax/_src/maps.py:627: note:     lower

bokeh (https://github.com/bokeh/bokeh)
+ src/bokeh/document/callbacks.py: note: In member "add_session_callback" of class "DocumentCallbackManager":
+ src/bokeh/document/callbacks.py:185:31: error: Incompatible types in assignment (expression has type "Callable[[], None]", variable has type "_Wrapped[[], None, [], None]")  [assignment]

ibis (https://github.com/ibis-project/ibis)
+ ibis/expr/types/relations.py:3037: error: Missing positional argument "predicates" in call to "__call__" of "_Wrapped"  [call-arg]

hydra-zen (https://github.com/mit-ll-responsible-ai/hydra-zen)
- src/hydra_zen/wrapper/_implementations.py:503: error: Incompatible types in assignment (expression has type "Zen[P, R]", variable has type "Callable[[Any], Any]")  [assignment]
+ src/hydra_zen/wrapper/_implementations.py:503: error: Incompatible types in assignment (expression has type "Zen[P, R]", variable has type "_Wrapped[P, R, [Any], Any]")  [assignment]
- src/hydra_zen/wrapper/_implementations.py:503: note: "Zen[P, R].__call__" has type "Callable[[DataClass_ | type[DataClass_] | dict[Any, Any] | DictConfig | str], R]"

[JelleZijlstra]

The bokeh hit looks unavoidable, the variable is assigned different types in two branches and mypy generally doesn't like that.

I suspect the ibis hit is because mypy thinks we have an unbound method wrapping a bound one (https://github.com/ibis-project/ibis/blob/7667328ada9cb8ef6bb4c456636c6a7ccab2ad1a/ibis/expr/types/joins.py#L274). Probably hard to avoid, but I'm also not sure why ibis needs the @wraps() call at all.

Overall, this feels like it's probably an improvement.

[tamird]

What do I need to do to get this landed?