Add static typing

[MarcSkovMadsen]

Closing #376

This PR adds static typing to param in a way that will not fundamentally change the current implementation and keep the api.

A more radical approach would be to leave descriptor pattern and engage with static typing stakeholders (including mypy) and get param supported in same way as dataclasses, attrs and pydantic.

Recommendation (from analysis.md)

• Adopt PEP 681’s @dataclass_transform for Param’s metaclass or base class.
• Provide and maintain high-quality stubs for Param descriptors.
• Recommend using class-specific stubs for full static typing, including init signatures.
• Avoid custom plugins unless absolutely necessary.
• Do not recommend wrapper functions with @overload as a general solution for Param static typing.

Update - Plugin

Ahh. I tried experimenting with a plugin to generate the __init__ signature - but this was too complicated for me and AI. See https://docs.pydantic.dev/latest/integrations/mypy/#enabling-the-plugin and https://github.com/pydantic/pydantic/blob/main/pydantic/mypy.py.

Todo

• [x] Define problem (see problem.md)
• [x] Analyze and experiment (see experiment.md and experiment.py. Run with mypy static-typing/experiment.py --no-incremental.)
• [x] Propose solution (see analysis.md)
• [ ] Collect feedback
• [ ] Get more contributors onboard. Technically I might be able to succeed. But there will be so many opinions about this that more people need to get deeply involved. We will have to take decissions.
• [ ] Implement, test and review.

[philippjfr]

Woah, this looks extremely promising.

[maximlt]

@MarcSkovMadsen I'm not really sure what I'm supposed to review here and what the actual proposal is.

[MarcSkovMadsen]

Review the recommendation/ proposal

Would you accept a PR like that? If yes then I will start implementing.

The .md documents provides the details and .py experiment file can enable you to try this hands on.

[philippjfr]

Thanks again for starting this @MarcSkovMadsen. After reviewing what's here I'm wavering a little bit. Effectively based on my reading, and until Python adopts type-aware descriptors we don't have many good options. With the list effectively being:

1. This approach which is fairly imprecise in the types, i.e. yes we can correctly type param.Integer but can't handle allow_None correctly. It's still an improvement but it's quite limited.
2. Combine approach 1. with manual type annotations, i.e. the descriptor is typed loosely but the user is asked to be more specific:

from typing import Optional

class Integer:
    def __get__(self, instance: object, owner: type) -> Optional[int]:
        ...
    def __set__(self, instance: object, value: Optional[int]) -> None:
        ...

class Model(param.Parameterized):
    a: int = Integer()
    b: Optional[int] = Integer(allow_None=True)

3. Using generics as in @hoxbro's PR https://github.com/holoviz/param/pull/985, would move the definition into the parameter declaration:

class Model(param.Parameterized):
    a = Integer[int]()
    b = Integer[Optional[int]](allow_None=True)

with some extra effort in defining TypeVar (which Simon already started playing with) with bound types you only have to define more restrictive types, i.e. it would simplify to:

class Model(param.Parameterized):
    a = Integer[int]()
    b = Integer(allow_None=True)

4. Create distinct Descriptor Types, i.e. have an OptionalInteger and an Integer parameter type. I think this is out by default because it fundamentally changes param's design.

On balance I think the approach using generics is most promising, even if it is somewhat verbose. So my feeling is that @hoxbro's PR https://github.com/holoviz/param/pull/985/files is the correct place to start from.

[maximlt]

Traitlets seems to have figured out something interesting, you can see that with allow_none=True the type of y is int | None. Certainly worth looking into their typing PRs https://github.com/ipython/traitlets/pulls?q=is%3Apr+typing+is%3Aclosed and implementation. I'm focused on finishing hvPlot's doc overhaul this summer and would very much like to dive into Param's typing story in September.

# t.py
from traitlets import HasTraits, Int

class P(HasTraits):
    x = Int(default=2)
    y = Int(allow_none=True)

p = P(x=3, y=None)

reveal_type(p.x)
reveal_type(p.y)

❯ mypy t.py
t.py:9: note: Revealed type is "builtins.int"
t.py:10: note: Revealed type is "Union[builtins.int, None]"