pydantic
pydantic copied to clipboard
pydantic
'Partial' Equivalent
Question
Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
pydantic version: 1.5.1
pydantic compiled: True
install path: /.../venv/lib/python3.7/site-packages/pydantic
python version: 3.7.4 (default, Sep 7 2019, 18:27:02) [Clang 10.0.1 (clang-1001.0.46.4)]
platform: Darwin-19.4.0-x86_64-i386-64bit
optional deps. installed: ['typing-extensions']
Hey Samuel & team,
First of all, huge fan of Pydantic, it makes working in Python so much better. I've been through the docs and many of the issues several times, and until this point I've found all the answers I was looking for. If I missed one in this case - sorry in advance!
Looking to be able to transform a model at runtime to another with the same fields but where all are Optional, akin to Typescript's Partial (hence the issue title). Something similar is discussed in part in at least one FastAPI issue, but since the answer there is basically "duplicate your model" and since this is inherently a pydantic thing, I thought I'd ask here.
By way of a little bit of hacking on model internals, I've been able to get the functionality I need for a relatively trivial use case:
from __future__ import annotations
from pydantic import BaseModel, create_model
from typing import Optional
class BaseSchemaModel(BaseModel):
# ...
@classmethod
def to_partial(cls) -> BaseSchemaModel:
return get_partial_model(cls)
def get_partial_model(model: BaseModel) -> BaseModel:
"""
Return a model similar to the original, but where all fields are Optional.
Note: this minimal implementation means that many Pydantic features will be discarded
(such as alias). This is not by design and could stand to improve.
"""
new_fields = {
name: (Optional[model.__annotations__.get(name)], model.__fields__[name].default or None)
for name in model.__fields__
}
return create_model("GeneratedPartialModel", **new_fields, __config__=model.__config__)
verified with a simple test:
import pytest
from pydantic import ValidationError
from typing import Optional
from app.utilities.pydantic import BaseSchemaModel
def test_partial_model():
class TestModel(BaseSchemaModel):
a: int
b: int = 2
c: str
d: Optional[str]
with pytest.raises(ValidationError):
TestModel()
model = TestModel(a=1, c='hello')
assert model.dict() == {'a': 1, 'b': 2, 'c': 'hello', 'd': None}
PartialTestModel = TestModel.to_partial()
partial_model = PartialTestModel()
assert partial_model.dict() == {'a': None, 'b': 2, 'c': None, 'd': None}
However this doesn't support much of Pydantic's functionality (aliases, for a start). In the interest of a better issue report and possibly a PR, I tried a couple other things, but in the time I allocated did not get them to work:
from pydantic import BaseModel, create_model
from pydantic.fields import ModelField
class Original(BaseModel):
a: int
b: str
c: Optional[str]
class PartialModel(BaseModel):
a: Optional[int]
b: Optional[str]
c: Optional[str]
def copy_model_field(field: ModelField, **kwargs) -> ModelField:
params = (
'name',
'type_',
'class_validators',
'model_config',
'default',
'default_factory',
'required',
'alias',
'field_info'
)
return ModelField(**{param: kwargs.get(param, getattr(field, param)) for param in params})
# Doesn't work - ModelField not acceptable in place of FieldInfo
GeneratedPartialModel = create_model('GeneratedPartialModel', **{name: copy_model_field(field, required=False) for name, field in Original.__fields__.items()})
# Doesn't work - field_info doesn't contain all the necessary information
GeneratedPartialModel = create_model('GeneratedPartialModel', **{name: copy_model_field(field, required=False).field_info for name, field in Original.__fields__.items()})
# This works for my use case - but without aliases and probably without some other functionality as well
new_fields = {name: (Optional[Original.__annotations__.get(name)], Original.__fields__[name].default or None) for name in Original.__fields__}
GeneratedPartialModel = create_model('GeneratedPartialModel', **new_fields)
Would be happy to put in a PR for this, if
a. it doesn't exist already
b. it would be useful
c. I knew where that would best fit - on ModelMetaclass? BaseModel? A utility function to_partial?
Thanks!
Generally I would make the Partial model a subclass of the Original:
from typing import Optional
from pydantic import BaseModel
class Original(BaseModel):
a: int
b: str
c: Optional[str]
class PartialModel(Original):
a: Optional[int]
b: Optional[str]
c: Optional[str]
which retains the validation and everything else about the original, but makes all the fields it defines Optional.
In : Original(a=1, b="two")
Out: Original(a=1, b='two', c=None)
In : PartialModel(b="three")
Out: PartialModel(a=None, b='three', c=None)
Sinc you already have some optional fields in your original, you don't need to redefine them, though it does help with explicitness:
In : class PartialModel(Original):
...: a: Optional[int]
...: b: Optional[str]
...:
In : PartialModel(b="four")
Out: PartialModel(a=None, b='four', c=None)
Thanks @StephenBrown2 - yeah, that’s what was suggested in the linked FastAPI issue and is what I’d consider the simplest approach. Simplicity is a plus but it has two important caveats:
- code duplication and vulnerability to drift
- not available at runtime, which is what I’m looking for in this case.
I would agree, though:
- code duplication and vulnerability to drift
- If you define your models next to each other, it will be more difficult to drift since you'd see the changes, though they would have to be made in two places, explicit is better than implicit.
- not available at runtime, which is what I’m looking for in this case.
- I'm not sure what you mean by "at runtime"? The model wouldn't change regardless if you defined it beforehand with the Base model or if you generated it from the Base model, would it? Or maybe I misunderstand.
Even with that said, I can see the benefit of a generated Partial model, for update requests for example. I also don't know the best way to go about that, though, so I'll defer to @samuelcolvin for his thoughts.
Hi @kalzoo, we're using a different method that does not require code duplication:
First define your Partial model without Optionals, but with a default value equal to a missing sentinel (https://github.com/samuelcolvin/pydantic/issues/1761). That should allow you to create objects where the supplied fields must validate, and where the omitted fields are equal to the sentinel value.
Then, create a Full model by subclassing the Partial model without redefining the fields, and adding a Config class that sets validate_all to True so that even omitted fields are validated.
hey I use this mixin to avoid deduplication:
class Clonable(BaseModel):
@classmethod
def partial(cls):
return cls.clone(to_optional='__all__')
@classmethod
def clone(
cls,
*,
fields: Set[str] = None,
exclude: Set[str] = None,
to_optional: Union[Literal['__all__'], Set[str], Dict[str, Any]] = None
) -> 'Clonable':
if fields is None:
fields = set(cls.__fields__.keys())
if exclude is None:
exclude = set()
if to_optional == '__all__':
opt = {f: None for f in fields}
opt.update(cls.__field_defaults__)
elif isinstance(to_optional, set):
opt = {f: None for f in to_optional}
opt.update(cls.__field_defaults__)
else:
opt = cls.__field_defaults__.copy()
opt.update(to_optional or {})
model = create_model(
cls.__name__,
__base__=Clonable,
**{
field: (cls.__annotations__[field], opt.get(field, ...))
for field in fields - exclude
}
)
model.__name__ += str(id(model))
return model
I needed this too. I have an implementation, but I think this is something that belongs in pydantic itself, and for that some more polish is needed. Does any maintainer have time to have a look and advise? Perhaps @dmontagu as it involves generics? The relevant commit is here: 4cbe5b0c339281f1f39c019fa27350f414ebf328
I was going to create a Pull Request, but apparently I'm supposed to open an issue to discuss first, and this issue already exists. If this is welcome (once polished up), I think opening a Pull Request to discuss the details makes sense. Is that what is expected?
Thanks for using pydantic. 🙏
As part of a migration to using discussions and cleanup old issues, I'm closing all open issues with the "question" label. 💭 🆘 🚁
I hope you've now found an answer to your question. If you haven't, feel free to start a :point_right: question discussion.
Humm, just seen this while reviewing #2245.
I've had this problem too, my solution is to use validate_model directly, see #932.
That way you get errors returned to you and can decide what to do with them. You could even then use construct() to create a model from your values.
The clone() solutions suggested above by @mortezaPRK should also work (I haven't looked through the code there).
You could easily build a utility function for this, either shipped with pydantic or standalone.
The main problem is that type hints and knowledge about the model are no longer valid. You thing you have an instance of (for example) User but actually email is None or raises an AttributeError.
That's one of the advantages of using my validate_model solution above, because you get back a dict, you're less tempted to pretend it's a full model.
Overall, I see the problem here, but I'm not yet sure I know what the best solution is.
Would love if a skip_fields option was provided which would do the same thing as validate_model. Would be a lot cleaner if I could pass it to my BaseModel class
Alternatively, passing the argument to the instance of my BaseModel class would be great as well
MyBaseModel(**the_model).dict(skip_invalid=True)
I had the need for this feature as well, and this is how I solved it:
from typing import *
class Partial(Generic[T]):
'''Partial[<Type>] returns a pydantic BaseModel identic to the given one,
except all arguments are optional and defaults to None.
This is intended to be used with partial updates.'''
_types = {}
def __class_getitem__(cls: Type[T], item: Type[Any]) -> Type[Any]:
if isinstance(item, TypeVar):
# Handle the case when Partial[T] is being used in type hints,
# but T is a TypeVar, and the type hint is a generic. In this case,
# the actual value doesn't matter.
return item
if item in cls._types:
# If the value was already requested, return the same class. The main
# reason for doing this is ensuring isinstance(obj, Partial[MyModel])
# works properly, and all invocation of Partial[MyModel] return the
# same class.
new_model = cls._types[item]
else:
class new_model(item):
'''Wrapper class to inherit the given class'''
for _, field in new_model.__fields__.items():
field.required = False
if getattr(field, 'default_factory'):
field.default_factory = lambda: None
else:
field.default = None
cls._types[item] = new_model
return new_model
Basically, the idea is to have Partial[MyModel] returning a copy of the model, where all the fields have been changed to optional with default to None.
I just created https://github.com/team23/pydantic-partial just to achieve this ;-)
This is still an early version, but is fully tested.
Hi all, thanks so much for your patience.
I've been thinking about this more, prompted partly by someone creating a duplicate issue, #5031.
I would love to support "partial-isation" of a pydantic model, or dataclass.
My proposal would b a function which creates a partial variant of a model
Usage would be something like this
from pydantic import BaseModel, partial, PydanticUndefined
class MyModel(BaseModel):
x: int
y: str
PartialModel1 = partial(MyModel)
PartialModel2 = partial(MyModel, missing_value=PydanticUndefined)
I know @dmontagu would prefer Partial[MyModel] to be closer to typescript, I'd be happy with that too although I think it's slightly less canonical.
The real point here is that ideally this would be added to python itself so static typing libraries recognised partial.
Therefore, when we get the time we should submit a proposal to discuss.python.org, and be prepare to write a PEP if/when we get enough interest.
We could also ask mypy/pyright to support this before the pep is accepted.
Lastly, we could even add support for partial to our mypy extension and thereby have full typing support for it without any input from others.
Obviously this will be pydantic V2 only, and probably won't be added until V2.1 or later, I don't think we'll have time to work on it before v2 is released, still, happy to hear input from others...?
Hi @samuelcolvin, I've done some modification to the function I proposed into #5031 .
The new code doesn't modify the original model and deals also with List, Dict and Union.
Tested with python 3.10.4 and pydantic 1.10.4
Known issues:
- when using pipe notation (from
types.UnionType) instead oftyping.Union(e.g.Engine: Fuel | str) the function returns error:TypeError: 'type' object is not subscriptable - cannot deal with pydantic types (e.g.
conlist) or more in general all type inpydantic.types
Do you have any suggestion for improving this implementation? How to deal with the two previous issues? We understand you prioritize V2 implementation but it would be nice to have a workaround for V1.
from pydantic import BaseModel, validator, create_model, ValidationError, class_validators
from typing import Type, Union, Optional, Any, get_origin, get_args
from pydantic.fields import ModelField
# Returns a new model class which is a copy of the input one
# having all inner fields and subfields as Optional
def make_optional(cls: Type[BaseModel]):
field_definitions: dict = {}
field: ModelField
validators_definitions: dict = {}
# exit condition: if 'cls' is not a BaseModel return the Optional version of it
# otherwise its fields must be optionalized
if not issubclass(cls, BaseModel):
return ( Optional[cls] )
for name, field in cls.__fields__.items():
# keep original validators but allow_reuse to avoid errors
v: class_validators.Validator
for k, v in field.class_validators.items() or []:
validators_definitions[k] = validator(
name,
allow_reuse=True, # avoid errors due the the same function owned by original class
always=v.always,
check_fields=v.check_fields,
pre=v.pre,
each_item=v.each_item
)(v.func)
# Compute field definitions for 'cls' inheriting from BaseModel recursively.
# If the field has an outer type (e.g. Union, Dict, List) then construct the outer type and trasform its inner arguments as Optional
# Otherwise the field can be either a BaseModel type (again) or a standard type (so just need to call the function recursively)
origin: Any | None = get_origin(field.outer_type_)
args = get_args(field.outer_type_)
if field.sub_fields and origin:
field_definitions[name] = ( (origin[ tuple( make_optional(arg_type) for arg_type in args ) ]), field.default or None )
else:
field_definitions[name] = ( make_optional(field.outer_type_), field.default or None )
return create_model(cls.__name__+"Optional", __base__=cls.__base__, __validators__=validators_definitions, **field_definitions)
class Fuel(BaseModel):
name: str
spec: str
class Engine(BaseModel):
type: str
fuel: Union[str, Fuel]
class Wheel(BaseModel):
id: int
name: str = "Pirelli"
@validator('id')
def check_id(cls, v):
if v <= 0:
raise ValueError('ID must be greater than 0')
return v
class Driver(BaseModel):
name: str
number: int
class Car(BaseModel):
name: str
driver: Driver
wheels: list[Wheel]
engine: Union[Engine, str]
# 0. Make Optional Car
CarOptional: Any | Type[None] = make_optional(Car)
print()
# 1. Create a standard Car
car1 = Car(name="Ferrari", driver=Driver(name='Schumacher', number=1), engine=Engine(type="V12", fuel=Fuel(name="Petrol",spec="XXX")), wheels=[{"id": 1}, {"id": 2}, {"id": 3}, {"id": 4}])
print("\n1) Standard Car \n", car1)
# 2. Create a CarOptional model having Optional fields also in nested objects (e.g. Fuel.name becomes Optional as well)
try:
car_opt1 = CarOptional(engine=dict(type="V12", fuel=dict(spec="XXX")), wheels=[{}, {"id": 2}, {"id": 3}, {"id": 4}])
except ValidationError as e:
assert False
except Exception as e:
assert False
else:
print("\n2) Optional Car1 with 'engine.fuel : Fuel \n", car_opt1)
assert True
try:
car_opt2 = CarOptional(driver=dict(name='Leclerc'), engine=dict(type="V12", fuel='ciao'), wheels=[{}, {"id": 2}, {"id": 3}, {"id": 4}])
except ValidationError as e:
assert False
except Exception as e:
assert False
else:
print("\n2) Optional Car2 with 'engine.fuel : str \n", car_opt2)
assert True
# 3. Validators are still executed for fields with a value (e.g. Wheel.id = 0)
print("\n3) Optional Car but WheelOptional.id not valid \n")
try:
car_not_valid = CarOptional(engine=dict(type="V12", fuel=dict(spec="XXX")), wheels=[{"id": 0}, {"id": 2}, {"id": 3}, {"id": 4}])
except ValidationError as e:
print(e)
assert True
else:
assert False
# 4. Must raise a validation error
print("\n4) Standard Car still working: should return error missing 'name', 'driver' and 'wheels' \n")
try:
car2: Car = Car(engine=Engine(type="V12", fuel=Fuel(name="Petrol",spec="XXX")))
except ValidationError as e:
print(e)
assert True
else:
assert False
After some reasoning and hard fighting on that topic, I ended up with un updated version of the previous recursive algorithm which handles for example types.UnionType, conlist and complex types (e.g. Engine | str | int | dict[str, Engine] ).
After testing it I'm almost satisfied but I see it is very complex and not 100% reliable. That's why for our project we finally decided to just make a new copy of the class definitions we need as Optional. Code duplication is not nice but at least we know it works.
Anyway I wanted to share my effort with the community cause it may be useful to someone else. Moreover, I've just understood that V2 should be very close (Q1 2023) so probably it doesn't make sense to put this effort for having a Typescript's Partial equivalent in V1. In V2 it may be also simpler to implement if the new modeling logic fits it in a better way (at least I hope so).
One thing I really don't understand in my code, is why the last test fails saying that function wheel.get_key() doesn't exist (AttributeError: 'WheelOptional' object has no attribute 'get_key'), while the validator is able to make use of the same get_key() function -_-
from pydantic import BaseModel, validator, create_model, ValidationError, class_validators, conlist
from typing import Type, Union, Optional, Any, List, Dict, Tuple, get_origin, get_args
from pydantic.fields import ModelField
from pydantic.class_validators import Validator
from types import UnionType
def print_check_model(cls):
print(f"\n------------------------------------ check '{cls}' model")
if issubclass(cls, BaseModel):
for field in cls.__fields__.values():
print("field", " ##### name '", cls.__name__+"."+field.name, "' ##### outer_type_", field.outer_type_, " ##### type_", field.type_, " ##### required", field.required, " ##### allow_none", field.allow_none)
if field.sub_fields:
for sub in field.sub_fields or [] :
print("sub", " ##### name '", cls.__name__+"."+sub.name, "' ##### outer_type_", sub.outer_type_, " ##### type_", sub.type_, " ##### required", sub.required, " ##### allow_none", sub.allow_none)
print_check_model(sub.type_)
else:
print(field.type_)
print_check_model(field.type_)
def copy_validator(field_name: str, v: Validator) -> classmethod:
return validator(
field_name,
allow_reuse=True, # avoid errors due the the same function owned by original class
always=v.always,
check_fields=v.check_fields,
pre=v.pre,
each_item=v.each_item
)(v.func)
# Returns a new model class which is a copy of the input one
# having all inner fields and subfields as non required
def make_optional(cls: Type[BaseModel], recursive: bool):
field_definitions: dict = {}
field: ModelField
validators_definitions: dict = {}
# if cls has args (e.g. list[str] or dict[int, Union[str, Driver]])
# then make optional types of its arguments
if get_origin(cls) and get_args(cls):
return get_origin(cls)[tuple(make_optional(arg, recursive) for arg in get_args(cls))]
# exit condition: if 'cls' is not a BaseModel return the Optional version of it
# otherwise its fields must be optionalized
if not issubclass(cls, BaseModel):
return cls
for name, field in cls.__fields__.items():
# keep original validators but allow_reuse to avoid errors
v: class_validators.Validator
for k, v in field.class_validators.items() or []:
validators_definitions[k] = copy_validator(name, v)
# Compute field definitions for 'cls' inheriting from BaseModel recursively.
# If the field has an outer type (e.g. Union, Dict, List) then construct the outer type and trasform its inner arguments as Optional
# Otherwise the field can be either a BaseModel type (again) or a standard type (so just need to call the function recursively)
origin: Any | None = get_origin(field.outer_type_)
args = get_args(field.outer_type_)
if not recursive:
field_definitions[name] = ( field.outer_type_, field.default or None )
elif origin in (dict, Dict, tuple, Tuple, list, List, Union, UnionType):
if origin is UnionType: # handles 'field: Engine | str'
origin = Union
field_definitions[name] = ( origin[ tuple( make_optional(arg_type, recursive) for arg_type in args ) ], field.default or None ) # type: ignore
# handle special cases not handled by previous if branch (e.g. conlist)
elif field.outer_type_ != field.type_:
if issubclass(field.outer_type_, list): # handles conlist
field_definitions[name] = ( list[ make_optional(field.type_, recursive) ], field.default or None )
else:
raise Exception(f"Case with outer_type_ {field.outer_type_} and type_ {field.type_} not handled!!")
else:
field_definitions[name] = ( make_optional(field.outer_type_, recursive), field.default or None )
return create_model(cls.__name__+"Optional", __config__=cls.__config__, __validators__=validators_definitions, **field_definitions)
# ____________________________________________________________________________________________________________________________________________________________
class Comp(BaseModel):
name: str
class Fuel(BaseModel):
name: str
spec: str
class Engine(BaseModel):
type: str
fuel: Union[str, Fuel]
eng_components: dict[str, list[Comp]]
eng_tuple_var: tuple[str, int, list[Comp]]
class Wheel(BaseModel):
id: int
name: str = "Pirelli"
def get_key(self) -> str:
return self.id
def __hash__(self) -> int:
return hash(self.get_key())
def __eq__(self, other) -> bool:
if not isinstance(other, Wheel):
return False
return self.get_key() == other.get_key()
@validator('id')
def check_id(cls, v):
if v <= 0:
raise ValueError('ID must be greater than 0')
return v
class Driver(BaseModel):
name: str
number: int
class Car(BaseModel):
class Config:
extra: str = "forbid"
name: str
driver: Driver
wheels: conlist(Wheel, unique_items=True)
engine: Engine | str | int | dict[str, Engine]
components: dict[str, list[Comp]]
tuple_var: tuple[str, int, list[Comp]]
# ____________________________________________________________________________________________________________________________________________________________________________
# 0. Make Optional Car
CarOptional: Any | Type[None] = make_optional(Car, True)
print()
print_check_model(CarOptional)
# 0. Create a standard Car
car0 = Car(name="Ferrari",
driver=Driver(name='Schumacher', number=1),
engine=Engine(type="V12", fuel=Fuel(name="Petrol",spec="XXX"), eng_components=dict({"c1": [{"name": "eng-comp1.1"}, {"name": "eng-comp1.2"}], "c2": [{"name": "eng-comp2.1"}, {"name": "eng-comp2.2"}]}), eng_tuple_var=tuple(["ciao", 34, [{"name": "tup-comp1"}, {"name": "tup-comp2"}]]) ),
wheels=[{"id": 1}, {"id": 2}, {"id": 3}, {"id": 4}],
components=dict({"c1": [{"name": "comp1.1"}, {"name": "comp1.2"}], "c2": [{"name": "comp2.1"}, {"name": "comp2.2"}]}),
tuple_var=tuple(["ciao", 34, [{"name": "comp1.1"}, {"name": "comp1.2"}]])
)
print("\n0) Standard Car \n", car0, "\n")
# 1. Create a CarOptional model having Optional fields also in nested objects (e.g. Fuel.name becomes Optional as well)
try:
car_opt1 = CarOptional(engine=dict(type="V12", fuel=dict(spec="XXX")), wheels=[{}, {"id": 2}, {"id": 3}, {"id": 4}])
print_check_model(CarOptional)
except ValidationError as e:
print("[KO] car_opt1")
assert False
except Exception as e:
print("[KO] car_opt1")
assert False
else:
print("\n[ok] 1) Optional Car1 with 'engine.fuel : Fuel\n", car_opt1)
assert True
# 2. Create a CarOptional model having Optional fields also in nested objects (e.g. Fuel.name becomes Optional as well)
try:
car_opt2 = CarOptional(driver=dict(name='Leclerc'), engine=dict(type="V12", fuel='ciao'), wheels=[{}, {"id": 2}, {"id": 3}, {"id": 4}])
except ValidationError as e:
print("[KO] car_opt2")
print(e)
assert False
except Exception as e:
print("[KO] car_opt2")
assert False
else:
print("\n[ok] 2) Optional Car2 with 'engine.fuel : str \n", car_opt2)
assert True
# 3. Validators are still executed for fields with a value (e.g. Wheel.id = 0)
print("\n3) Optional Car but WheelOptional.id = 0 not valid \n")
try:
car_not_valid = CarOptional(engine=dict(type="V12", fuel=dict(spec="XXX")), wheels=[{"id": 0}, {"id": 2}, {"id": 3}, {"id": 4}])
except ValidationError as e:
print("[ok] car_not_valid ")
print(e)
assert True
else:
print("[KO] car_not_valid")
assert False
# 4. Validators are still executed for conlist with duplicated items (e.g. having two items with Wheel.id = 2)
print("\n4) Optional Car but duplicated WheelOptionals with same 'id = 2', is not valid due to 'id' validator\n")
try:
car_not_valid = CarOptional(engine=dict(type="V12", fuel=dict(spec="XXX")), wheels=[{"id": 1}, {"id": 2}, {"id": 2}, {"id": 4}])
except ValidationError as e:
print("[ok] car_not_valid ")
print(e)
assert True
else:
print("[KO] car_not_valid")
assert False
# 5. Standard Car must still raise a validation error
print("\n5) Standard Car still working: should return error missing 'name', 'driver', 'wheels', 'components' and 'tuple_var' \n")
try:
car2: Car = Car(engine=Engine(type="V12", fuel=Fuel(name="Petrol",spec="XXX")))
except ValidationError as e:
print("[ok] car2 ")
print(e)
assert True
else:
print("[KO] car2")
assert False
# 6. Check inner Driver model not modified
print("\n6) Inner Driver model not modified \n")
try:
print("driver", Driver(number=12))
except ValidationError as e:
print("[ok] Driver ")
print(e)
assert True
else:
print("[KO] Driver")
assert False
CarOptionalNonRecursive = make_optional(Car, False)
# 7. Non recursive Optional Car 1
print("\n7) Non recursive Optional Car 1 \n")
try:
caroptnonrec: CarOptionalNonRecursive = CarOptionalNonRecursive(driver=dict(number=12))
except ValidationError as e:
print("[ok] caroptnonrec1 ")
print(e)
assert True
else:
print("[KO] caroptnonrec1")
print(caroptnonrec)
assert False
# 8. Non recursive Optional Car 2
print("\n8) Non recursive Optional Car 2 \n")
try:
caroptnonrec: CarOptionalNonRecursive = CarOptionalNonRecursive(name='test')
except ValidationError as e:
print("[KO] caroptnonrec2")
print(e)
assert False
else:
print("[ok] caroptnonrec2 ")
print(caroptnonrec)
assert True
# 9. Complex Tuple
print("\n9) Complex Tuple \n")
try:
caropttuple: CarOptional = CarOptional(name='test', tuple_var=tuple(["ciao", 34, [{}, {"name": "comp1.2"}]]), engine=dict(type="V12", fuel="fuelXX", eng_components=dict({"c1": [{}, {"name": "eng-comp1.2"}], "c2": [{"name": "eng-comp2.1"}, {"name": "eng-comp2.2"}]}), eng_tuple_var=tuple(["ciao", 34, [{}, {"name": "tup-comp2"}]]) ), )
except ValidationError as e:
print("[KO] caropttuple")
print(e)
assert False
else:
print("[ok] caropttuple ")
print(caropttuple)
assert True
# 10. Additional forbidden param
print("\n10) Additional forbidden param \n")
try:
caroptadditional: CarOptional = CarOptional(additional="non_existing" )
except ValidationError as e:
print("[ok] caroptadditional ")
print(e)
assert True
else:
print("[KO] caroptadditional")
print(caroptadditional)
assert False
# 11. Engine of type int but why wheel.get_key() not found?!?!?
print("\n11) Engine of type 'int' but why wheel.get_key() not found?!?!? \n")
try:
car_opt3 = CarOptional(driver=dict(name='Leclerc'), engine=2, wheels=[{}, {"id": 2}, {"id": 3}, {"id": 4}])
except ValidationError as e:
print("[KO] car_opt3")
print(e)
assert False
except Exception as e:
print("[KO] car_opt3")
assert False
else:
print("\n[ok] Optional Car3 with 'engine : int \n", car_opt3)
for w in car_opt3.wheels:
print(w.get_key())
assert True
While I think we would like to better support this, ultimately there just isn't anything super analogous to typescript's Partial in the python typing system. As such, we've decided not to support a Partial-like feature for the time being, as we would rather wait until such a feature is present for dataclass or TypedDict and match the implementation, instead of coming up with our own.
I think we'd be open to revisiting this if a good proposal was brought forward, especially one that was compatible with type-checkers, etc. But for now I'm going to close this as "not planned" — at least until the language better supports these kinds of (more advanced) utility types.
See https://github.com/pydantic/pydantic/issues/830#issuecomment-1522202953 for a related comment.
I adapted a Partial Model that I found on StackOverflow for Pydantic V1x to work well with Pydantic V2.
from copy import deepcopy
from typing import Any, Callable, Optional, Type, TypeVar
from pydantic import BaseModel, create_model
from pydantic.fields import FieldInfo
Model = TypeVar("Model", bound=Type[BaseModel])
def partial_model(without_fields: Optional[list[str]] = None) -> Callable[[Model], Model]:
"""A decorator that create a partial model.
Args:
model (Type[BaseModel]): BaseModel model.
Returns:
Type[BaseModel]: ModelBase partial model.
"""
if without_fields is None:
without_fields = []
def wrapper(model: Type[Model]) -> Type[Model]:
base_model: Type[Model] = model
def make_field_optional(field: FieldInfo, default: Any = None) -> tuple[Any, FieldInfo]:
new = deepcopy(field)
new.default = default
new.annotation = Optional[field.annotation]
return new.annotation, new
if without_fields:
base_model = BaseModel
return create_model(
model.__name__,
__base__=base_model,
__module__=model.__module__,
**{
field_name: make_field_optional(field_info)
for field_name, field_info in model.model_fields.items()
if field_name not in without_fields
},
)
return wrapper
How use
class MyFullModel(BaseModel):
name: str
age: int
relation_id: int
@partial_model(without_fields=["relation_id"])
class MyPartialModel(MyFullModel):
pass
Here Is my working solution. Similar to some of the solutions above except that it works recursively and is able to make all sub schemas partial aswell.
from typing import Any, Optional, Type, Union, get_args, get_origin, Annotated
from pydantic import BaseModel, create_model
from pydantic.fields import FieldInfo
MaybePydantic = Type[Union[Any, BaseModel]]
def create_optional_field(field: Union[FieldInfo, MaybePydantic]) -> object:
field_type = field.annotation if isinstance(field, FieldInfo) else field
if origin := get_origin(field_type):
if origin is Annotated:
return Optional[field_type]
args = get_args(field_type)
optional_args = [Optional[create_optional_field(arg)] for arg in args]
return Optional[origin[*optional_args]]
# Handle BaseModel subclasses
if field_type and issubclass(field_type, BaseModel):
return Optional[Union[field_type, create_optional_model(field_type)]]
return Optional[field_type]
def create_optional_model(model: Type[BaseModel]) -> Type[BaseModel]:
"""
Make all fields in a pydantic model optional. Sub schemas will also become 'partialized'
"""
return create_model( # type: ignore
model.__name__ + "Optional",
__base__=model,
**{
name: (create_optional_field(field), None)
for name, field in model.model_fields.items()
},
)
Usage example:
class Bar(BaseModel):
id: str
class Baz(BaseModel):
items: List[Dict[str, Optional[Bar]]]
class Foo(BaseModel):
bar: Optional[HttpUrl]
baz: Baz
optional_cls = create_optional_model(Foo)
optional_cls()
For anyone interested in this feature, I'd love to support it, but really we need a Partial type in Python itself, so we can have a typesafe Partial implementation.
There's currently a discussion on discuss.python.org about adding a Partial type for typed dicts, feel free to upvote the original proposal and/or my reply.
For anyone interested in this feature, I'd love to support it, but really we need a
Partialtype in Python itself, so we can have a typesafePartialimplementation.There's currently a discussion on
discuss.python.orgabout adding aPartialtype for typed dicts, feel free to upvote the original proposal and/or my reply.
Nice!! I'll leave my 2 cents contribution there too.
@satheler Thank you for the snippet. Just want to point out that this part of the function will strip the resulting partial model of all validators, serializers, and private attributes that would've otherwise been inherited from the input model:
if without_fields:
base_model = BaseModel
I made some modifications that seem to be working. The idea is to still pass all fields to create_model whether they are in without_fields or not, but to wrap the annotations of the ones in without_fields with ClassVar, which will cause Pydantic to ignore them. This means we can now always inherit the original input model for __base__. However, pydantic will error if validators exist on non-existing fields, so to fix this I make a deepcopy of the input model, and set all validators or serializers to have the "check_fields" argument to false. This works whether the validator was created via a decorator or not, despite the name.
from copy import deepcopy
from dataclasses import asdict
from typing import Any, Callable, Optional, Type, TypeVar, ClassVar
from pydantic import BaseModel, create_model
from pydantic.fields import FieldInfo
Model = TypeVar("Model", bound=Type[BaseModel])
def partial_model(
without_fields: Optional[list[str]] = None,
) -> Callable[[Model], Model]:
"""A decorator that create a partial model.
Args:
model (Type[BaseModel]): BaseModel model.
Returns:
Type[BaseModel]: ModelBase partial model.
"""
if without_fields is None:
without_fields = []
def wrapper(model: Type[Model]) -> Type[Model]:
def make_field_optional(
field: FieldInfo, default: Any = None, omit: bool = False
) -> tuple[Any, FieldInfo]:
new = deepcopy(field)
new.default = default
new.annotation = Optional[field.annotation]
# Wrap annotation in ClassVar if field in without_fields
return ClassVar[new.annotation] if omit else new.annotation, new
model_copy = deepcopy(model)
# Pydantic will error if validators are present without the field
# so we set check_fields to false on all validators
for dec_group_label, decs in asdict(model_copy.__pydantic_decorators__).items():
for validator in decs.keys():
decorator_info = getattr(
getattr(model.__pydantic_decorators__, dec_group_label)[validator],
"info",
)
if hasattr(decorator_info, "check_fields"):
setattr(
decorator_info,
"check_fields",
False,
)
return create_model(
model.__name__,
__base__=model,
__module__=model.__module__,
**{
field_name: make_field_optional(
field_info, omit=(field_name in without_fields)
)
for field_name, field_info in model.model_fields.items()
},
)
return wrapper
