typer
typer copied to clipboard
tiangolo
[FEATURE] Support Pydantic Models as ParamTypes
Is your feature request related to a problem
I would like to use Pydantic Models as type annotations in my Typer CLI program.
The solution you would like
Typer would call Model.parse_raw on the string that was passed to the CLI
Describe alternatives you've considered
Rather than having Typer be so aware of the Pydantic API, I tried to create a custom click.ParamType that did the parsing, but even that did not work as custom types do not currently seem to be supported in the get_click_type function.
Additional context
Here's a simple example:
#!/usr/bin/env python3
import click
import typer
from pydantic import BaseModel
class User(BaseModel):
id: int
name = "Jane Doe"
class UserParamType(click.ParamType):
def convert(self, value, param, ctx):
return User.parse_raw(value)
USER = UserParamType()
def main(num: int, user: USER):
print(num, type(num))
print(user, type(user))
if __name__ == "__main__":
typer.run(main)
This currently throws:
Traceback (most recent call last):
File "./typer_exp.py", line 26, in <module>
typer.run(main)
File "/usr/local/lib/python3.7/site-packages/typer/main.py", line 848, in run
app()
File "/usr/local/lib/python3.7/site-packages/typer/main.py", line 213, in __call__
return get_command(self)()
File "/usr/local/lib/python3.7/site-packages/typer/main.py", line 238, in get_command
click_command = get_command_from_info(typer_instance.registered_commands[0])
File "/usr/local/lib/python3.7/site-packages/typer/main.py", line 422, in get_command_from_info
) = get_params_convertors_ctx_param_name_from_function(command_info.callback)
File "/usr/local/lib/python3.7/site-packages/typer/main.py", line 403, in get_params_convertors_ctx_param_name_from_function
click_param, convertor = get_click_param(param)
File "/usr/local/lib/python3.7/site-packages/typer/main.py", line 656, in get_click_param
annotation=main_type, parameter_info=parameter_info
File "/usr/local/lib/python3.7/site-packages/typer/main.py", line 586, in get_click_type
raise RuntimeError(f"Type not yet supported: {annotation}") # pragma no cover
RuntimeError: Type not yet supported: <__main__.UserParamType object at 0x7fc346234b10>
A major bonus would be if I could write the user: User type annotation directly, without creating the UserParamType.
Also - just want to say thank you for writing such an awesome python package and keep up the great work! 👏
I just noticed that this is very similar to #77. I will leave this open and change the name to specifically request support for Pydantic in order to differentiate it.
As proof of concept of how support for Pydantic Models could be implemented, I built on the monkey-patched version of get_click_type that @ananis25 wrote in #77:
#!/usr/bin/env python3
from typing import Any
import click
import pydantic
import typer
_get_click_type = typer.main.get_click_type
def supersede_get_click_type(
*, annotation: Any, parameter_info: typer.main.ParameterInfo
) -> click.ParamType:
if hasattr(annotation, "parse_raw"):
class CustomParamType(click.ParamType):
def convert(self, value, param, ctx):
return annotation.parse_raw(value)
return CustomParamType()
else:
return _get_click_type(annotation=annotation, parameter_info=parameter_info)
typer.main.get_click_type = supersede_get_click_type
class User(pydantic.BaseModel):
id: int
name = "Jane Doe"
def main(num: int, user: User):
print(num, type(num))
print(user, type(user))
if __name__ == "__main__":
typer.run(main)
$ ./typer_demo.py 1 '{"id": "2"}'
1 <class 'int'>
id=2 name='Jane Doe' <class '__main__.User'>
Also, Typer could settle on an API for custom types (e.g. in keeping with Pydantic, the Typer docs could declare "all custom ParamTypes must implement a parse_raw method") and this would then cover the use-case requested in #77 without needing to implement a registration process.
Oh, I didn't know about the parse_raw method provided by pydantic. This is definitely neater than keeping a global container of custom datatypes!
The only reason to prefer a registration api imo is that the code not using pydantic types/classes can also be used in a CLI without making any changes to it.
For ex - the typer app.command function also only registers the function being decorated instead of wrapping it, which is useful if the method hello is defined somewhere you don't want to make edits.
import typer
def hello(name: str):
return f"Hello {name}"
if __name__ == "__main__":
app = typer.Typer()
app.command()(hello)
+1
I really like the idea of using Pydantic models.
Another approach to parse_raw could be to traverse Pydantic models until we reach a supported type.
Parameters of type BaseModel could be parsed into multiple cli options like for example the traefik cli does.
This does only work for options and not for arguments.
#!/usr/bin/env python3
import click
import pydantic
import typer
class User(pydantic.BaseModel):
id: int
name: str = "Jane Doe"
def main(num: int, user: User):
print(num, type(num))
print(user, type(user))
if __name__ == "__main__":
typer.run(main)
The example above could then be called like this:
$ ./typer_demo.py 1 --user.id 2 --user.name "John Doe"
1 <class 'int'>
id=2 name='John Doe' <class '__main__.User'>
I was about to write a ticket along the lines of @Patdue's comment above, I would love something along those lines; I currently have several methods with large, almost identical signatures, which map to a pydantic model.
Something like that would be fantastic!
Another approach to
parse_rawcould be to traverse Pydantic models until we reach a supported type. Parameters of typeBaseModelcould be parsed into multiple cli options like for example the traefik cli does. This does only work for options and not for arguments.
I'm working on a pull request to support this behaviour. Because there is no longer a 1-1 mapping between the typed function parameters and the click.Parameters, it involves quite a lot of changes.
Btw. @tiangolo how do you feel about this feature?
@tiangolo can we get this green-lighted for a pull request?
@jackric better to ask for forgiveness than permission 😉
This would also help many other use cases:
- I wanted to use, my own formatting for
datetime, ex:today %H:%M,%M:%H(automatically uses date as today) - typer would be able to serialize any custom object
It also doesn't seem, it would require a lot of changes on how typer works internally to implement this. There is already PR #304
Also would love to see this! Typer looks awesome, but I like to manage my configs with Pydantic, and I don't want to describe all my params twice. So currently looking at https://github.com/SupImDos/pydantic-argparse or https://pypi.org/project/pydantic-cli/ instead. But, alas, argparse...
Btw, just found this: https://medium.com/short-bits/typer-the-command-line-script-framework-you-should-use-de9d07109f54 🤔
Going to try and implement this in #630, more or less exactly as described by @pypae
@tiangolo do you have a position on this feature request? If it were implemented well, would you accept? What might your definition of "implemented well" be?
An integration of typer and pydantic would be totally awesome! Currently one needs to write an annoying amount of boilerplate to map from CLI arguments to model fields. In an ideal solution, one could bind pydantic fields directly to cli params e.g. using some extra type hint inside Annotated (that way it would not interfere with the pydantic field config).
My dream solution would allow to have a well-integrated solution to have a common data model filled from various sources, such as:
- CLI arguments (via typer)
- config files (via pydantic)
- interactive user inputs (via rich)
For the latter part I'm using currently this approach to prompt fields for my config model:
from rich import print
from rich.panel import Panel
from rich.prompt import Confirm, Prompt
class MyBaseModel(BaseModel):
"""Tweaked BaseModel to manage the template settings.
Adds functionality to prompt user via CLI for values of fields.
Assumes that all fields have either a default value (None is acceptable,
even if the field it is not optional) or another nested model.
This ensures that the object can be constructed without being complete yet.
"""
model_config = ConfigDict(
str_strip_whitespace=True, str_min_length=1, validate_assignment=True
)
def check(self):
"""Run validation on this object again."""
self.model_validate(self.model_dump())
@staticmethod
def _unpack_annotation(ann):
"""Unpack an annotation from optional, raise exception if it is a non-trivial union."""
o, ts = get_origin(ann), get_args(ann)
is_union = o is Union
fld_types = [ann] if not is_union else [t for t in ts if t is not type(None)]
ret = []
for t in fld_types:
inner_kind = get_origin(t)
if inner_kind is Literal:
ret.append([a for a in get_args(t)])
elif inner_kind is Union:
raise TypeError("Complex nested types are not supported!")
else:
ret.append(t)
return ret
def _field_prompt(self, key: str, *, required_only: bool = False):
"""Interactive prompt for one primitive field of the object (one-shot, no retries)."""
fld = self.model_fields[key]
val = getattr(self, key, None)
if required_only and not fld.is_required():
return val
defval = val or fld.default
prompt_msg = f"\n[b]{key}[/b]"
if fld.description:
prompt_msg = f"\n[i]{fld.description}[/i]{prompt_msg}"
ann = self._unpack_annotation(fld.annotation)
fst, tail = ann[0], ann[1:]
choices = fst if isinstance(fst, list) else None
if fst is bool and not tail:
defval = bool(defval)
user_val = Confirm.ask(prompt_msg, default=defval)
else:
if not isinstance(defval, str) and defval is not None:
defval = str(defval)
user_val = Prompt.ask(prompt_msg, default=defval, choices=choices)
setattr(self, key, user_val) # assign (triggers validation)
return getattr(self, key) # return resulting parsed value
def prompt_field(
self,
key: str,
*,
recursive: bool = True,
missing_only: bool = False,
required_only: bool = False,
) -> Any:
"""Interactive prompt for one field of the object.
Will show field description to the user and pre-set the current value of the model as the default.
The resulting value is validated and assigned to the field of the object.
"""
val = getattr(self, key)
if isinstance(val, MyBaseModel):
if recursive:
val.prompt_fields(
missing_only=missing_only,
recursive=recursive,
required_only=required_only,
)
else: # no recursion -> just skip nested objects
return
# primitive case - prompt for value and retry if given invalid input
while True:
try:
# prompt, parse and return resulting value
return self._field_prompt(key, required_only=required_only)
except ValidationError as e:
print()
print(Panel.fit(str(e)))
print("[red]The provided value is not valid, please try again.[/red]")
def prompt_fields(
self,
*,
recursive: bool = True,
missing_only: bool = False,
required_only: bool = False,
exclude: List[str] = None,
):
"""Interactive prompt for all fields of the object. See `prompt_field`."""
excluded = set(exclude or [])
for key in self.model_fields.keys():
if missing_only and getattr(self, key, None) is None:
continue
if key not in excluded:
self.prompt_field(
key,
recursive=recursive,
missing_only=True,
required_only=required_only,
)
So some instance can be completed interactively using obj.prompt_field("key_name") to ask for a single field value or obj.prompt_fields(...) to walk through all (or configurable to ask only for missing values).
Would this somehow make sense inside of typer as well (because the intended use-case strongly correlates with use-cases for typer), or would it make sense to create a little library for that?
