python
python copied to clipboard
mkdocstrings
feature: Convert `Unpack[TypedDict]` to regular arguments
Is your feature request related to a problem? Please describe.
Not a problem, but when writing functions/classes with lots of options, and in particular when these options need to be forwarded down to a few levels, it is extremely convenient to use Unpack[TypeDict] as the type for keyword arguments and group all arguments in a class that can be reused.
For example:
from typing import TypedDict, Unpack
class Args(TypedDict):
"""Arguments for the add function"""
a: int
"""The first number"""
b: int
"""The second number"""
def add(**args: Unpack[Args]) -> int:
"""Add two numbers together.
Args:
**args: The arguments for the function.
Returns:
The sum of the two numbers.
"""
return args["a"] + args["b"]
Renders as:
Describe the solution you'd like
It would be nice if it were rendered as regular arguments instead, like:
Describe alternatives you've considered
They are basically above.
Additional context
I know this might be tricky, specially considering cases using Args(TypeDict, total=False) or NotRequired as in this case some arguments could not be present at all, which is different from passing None for example. But maybe there is a low hanging fruit, and at least some limited support could be added.
Thanks for the request @llucax! Related: https://github.com/mkdocstrings/griffe/issues/284.
Oh, thanks for the reference, I completely missed it. I was actually in doubt if it had to be reported in griffe or not :shrug:
No worries, it's not a duplicate. Support for Unpack must be implemented in Griffe, but here we also want an option to actually unpack the signature when rendering :slightly_smiling_face:
By the way, what about total=False typed dicts? Every parameter is optional, but doesn't have a default value. How do we store that? How do we render it? For now the feature puts non-optional parameters in the signature but it will cause issues when checking for breaking changes. I thought option 3 from https://github.com/mkdocstrings/griffe/issues/284 was more practical, but I'm leaning towards option 5 now.
Yeah I'll update the feature and go with option 5, adding logic for total=True/False, typing.Required and typing.NotRequired. Non-required parameters will be swallowed by **kwargs and documented in an "Other parameters" section.
Erm, no option is satisfying IMO. Expanding the signature is always incorrect (except when all parameters are required and no extra items are allowed). What do IDEs do? I think the most correct thing would be to expand all parameters, but add a default value like MISSING or NOT_REQUIRED to non-required ones. We'd only add back **kwargs if extra_items is specified. But then, what's the description of kwargs? Setting an english-hardcoded description seems ugly (unless we make upstream tools like mkdocstrings translate it, but still ugly). Also we must find a non-colliding name for kwargs (which could be taken by the typed dict already).
What do you want to show to your users? What do your users want to see instead of **kwargs: Unpack[Thing]?
Would it be enough if we just generate an "Other parameters" section without touching to signatures?
I was also thinking that maybe the best way to document non-required fields with no default with some sort of sentinel default. From the docs PoV that looks like enough, but I'm not sure how this would work in griffe, as I guess it is doing some AST transformations or something like that.
What do IDEs do?
Here is an example of what pyright does (in vim)
So it uses ... as a sentinel for non required, which is not correct but I find reasonable. It also doesn't rewrite the docstring, but I think mkdocstrings should.
I think the most correct thing would be to expand all parameters, but add a default value like MISSING or NOT_REQUIRED to non-required ones.
This seems to be what pyright does.
We'd only add back
**kwargsifextra_itemsis specified. But then, what's the description of kwargs? Setting an english-hardcoded description seems ugly (unless we make upstream tools like mkdocstrings translate it, but still ugly). Also we must find a non-colliding name for kwargs (which could be taken by the typed dict already).
What is extra_items?
What do you want to show to your users? What do your users want to see instead of **kwargs: Unpack[Thing]?
I find what pyright does reasonable. The only problem would be if the user accepts ... as a value that is different from the value being missing, and/or if they use ... as default for required fields, that would introduce an ambiguity (is this ... because it is an explicit default or is it because the field is not required). I think in practice ... should be semantically as missing, so using it for something else would be unexpected anyways, and this can happen with many other type hints, where type-checking can pass but at runtime you could do whatever you want and break all guarantees.
Thanks! I didn't think of using the ellipsis, sounds like a good idea 🙂
What is extra_items?
https://typing.python.org/en/latest/spec/typeddict.html#extra-items-and-closed-typeddicts
The only problem would be if the user accepts
...as a value that is different from the value being missing
Let's just say it won't happen 🙈 One would have to import EllipsisType from types to set it as valid type for a typed dict attr, so it's not that straightforward.
and/or if they use ... as default for required fields
You can't set defaults in typed dicts, so we're good here.
Thanks again!
typing.python.org/en/latest/spec/typeddict.html#extra-items-and-closed-typeddicts
I see, I thought I saw it before somewhere, but it is not mentioned at all in any of these 🤔 :
- https://docs.python.org/3/library/typing.html#typing.TypedDict
I guess because is a new unreleased feature, right?
But OK, with that it gets trickier. I guess one option is to put **kwargs: <value_of_extra_items> in the signature but not in the arguments documentation (or leave that documentation empty), as there is really no documentation for it. You could use __kwargs to make it more obvious it is just an "internal" artifact and minimize the chances of a name collision.
Ah right it's for 3.15 if I read the PEP metadata correctly.
Thanks for the suggestions, I'll experiment.
I went ahead with expanding to only named arguments, for both required and non-required fields, while non-required parameters get a ... default value. That's instead of adding **kwargs for non-required parameters.
The extra_items and closed features of typed dicts are not yet supported, we'll add support later when people request it (these are features for Python 3.15).