Pydantic Partial Updates do not work with Nested Objects

[rohitmusti]

Huge thank you for the maintainers that do so much thankless work to support this pretty cool piece of tech!

Checks

• [x] I added a descriptive title to this issue
• [x] I have searched (google, github) for similar issues and couldn't find anything
• [x] I have read and followed the docs and still think this is a bug

Bug

Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":

pydantic version: 1.9.1
pydantic compiled: True
             install path: /Users/rohitmusti/Library/Caches/pypoetry/virtualenvs/work-env/lib/python3.8/site-packages/pydantic
           python version: 3.8.13 (default, Jun  9 2022, 13:47:30)  [Clang 13.1.6 (clang-1316.0.21.2.5)]
                     platform: macOS-12.4-x86_64-i386-64bit
     optional deps. installed: ['dotenv', 'email-validator', 'typing-extensions']

Minimum Reproducible Example

from typing import List, Union
from pydantic import BaseModel
from enum import Enum

class PriceEnum(str, Enum):
    us = "USD",
    eu = "EU"


class Price(BaseModel):
    price: int = 5  
    currency: PriceEnum = PriceEnum.us


class Item(BaseModel):
    name: str = "default_name"
    description: str = "default_description"
    price: Price = Price()


items = {
    "foo": {"name": "Foo", "price": {"price": 2}},
}

from_db_item = Item(price=Price(price=3, currency = PriceEnum.eu)) 

request_model = Item(**items['foo'])  # a partial update that might come through a patch request

update_data = request_model.dict(exclude_unset=True)
# Have to deconstruct then reconstruct for pydantic parsing to work well enough to detect the sub-objects and not replace them w/ dicts
updated_item = Item.parse_obj(from_db_item.copy(update=update_data).dict())  

print("original")
print(from_db_item)

print("\nupdated:")
print(updated_item)
print(updated_item.price.price)  

Output Expectations vs Reality

### expected output
# original
# name='default_name' description='default_description' price=Price(price=3, currency=<PriceEnum.eu: 'EU'>)
# updated:
# name='Foo' description='default_description' price=Price(price=2, currency=<PriceEnum.: 'EU'>)   

### actual output
# original
# name='default_name' description='default_description' price=Price(price=3, currency=<PriceEnum.eu: 'EU'>)
# updated:
# name='Foo' description='default_description' price=Price(price=2, currency=<PriceEnum.us: 'USD'>)

[rohitmusti]

TLDR, the update param on copy is supposed to be able to handle a dictionary of values to change when updating a model here. It will reset the fields of sub objects instead of doing a partial update on the sub objects

[xkortex]

I believe this construction results in the intended behavior:

updated_item = from_db_item.copy(update={'price':from_db_item.price.copy(update=request_model.dict(exclude_unset=True))})

# Item(name='default_name', description='default_description', price=Price(price={'price': 2}, currency=<PriceEnum.eu: 'EU'>, name='Foo'))

That's obviously a bit clunky, and quickly becomes clunkier the deeper the model gets. I also think https://github.com/samuelcolvin/pydantic/issues/3785 might be related.

There exist workarounds, but patching/updating/recursive merge is a pretty common operation, so it would be highly convenient if the pydantic library supported this directly, either with a helper function or an extension to the copy method.

[rohitmusti]

really interesting approach! I agree that something supported directly would make this really useful. The issue is that my context has multiple layers of nested objects and if I needed to hand check and create these sort of partial updates for all of them based on their presence in a partial update, it would get very unwieldy

[whatamithinking]

Could something like this be added as a method to merge in the updates?

[anilsg]

Note this issue seems to be dealing with the same thing, inability of BaseModel.copy to treat nested sub-objects in update as objects rather than overwriting the entire sub-object with the dict: https://github.com/pydantic/pydantic/issues/3785

[Ezbaze]

Is this related? i have this weird beheviour when setting values on an already created Model, compared to the correct behaviour on the one where the dict is unpacked when creating the Model?

from typing import List, Optional
from pydantic import BaseModel


class Sub(BaseModel):
    name: str
    id: int


class Top(BaseModel):
    attributes: Optional[List[Sub]]


dummy_data = {
    "attributes": [
        {"name": "Sub Model 1", "id": 1},
        {"name": "Sub Model 2", "id": 2},
    ]
}

_modelTop1 = Top(**dummy_data)

_modelTop2 = Top()
_modelTop2.attributes = dummy_data["attributes"]

print(_modelTop1)
"""
attributes=[Sub(name='Sub Model 1', id=1), Sub(name='Sub Model 2', id=2)]
"""
print(_modelTop2)
"""
attributes=[{'name': 'Sub Model 1', 'id': 1}, {'name': 'Sub Model 2', 'id': 2}]
"""

[grub3r-codes]

I'm having a similar issue @Ezbaze:

def test_nested_model_update():
    class A(BaseModel):
        attr_: int

        @property
        def get_attr_(self):
            return self.attr_ * 2

    class B(BaseModel):
        list_: Optional[List[A]] = None

    b_1 = B(list_=[A(attr_=1)])
    b_2 = B()
    b_3 = b_2.copy(update=b_1.dict())
    b_3.list_[0].get_attr_

Here b_3.list[0] is saved as a dict instead of type A. This throws an error that dict does not have property getattr_

[sydney-runkle]

We’re no longer actively developing Pydantic V1 (although it will continue to receive security fixes for the next year or so), so we’re closing issues solely related to V1. If you really think this shouldn’t be closed, please comment or create a new issue 🚀.