[BUG] __root__ Models not working with Pydantic V2

[innicoder]

Describe the bug We had used previous version of Beanie and upgraded to V2, meanwhile the __root__ models are not longer supported in pydantic v2, only RootModel which basically means that all of the data that we had saved isn't compatible.

To Reproduce (our database no longer works after upgrading from v1 to v2)

class Grade(BaseModel):
    level: int

class GradeHistory(BaseModel):
    __root__: List[Grade]

class Student(Document):
    name: str
    age: int
    grades: GradeHistory

Expected behavior We should be able to migrate GradeHistory to RootModel and it should work perfectly with beanie but there isn't any support, this doesn't seem to be taken care of.

Additional context None

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity.

[GuyGooL5]

Pydantic V2 introduced TypeAdapter and annotated types with validators. Using RootModel feels a little anti-pattern in V2, maybe consider working with TypeAdapter more often,

In your example (and given you don't directly access GradeHistory you could define it as such

GradeHistory = List[Grade]

now it has become a type and is still validated in the context of a BaseModel and its descendants (View and Document)

---

In a case where you need to validate GradeHistory out of a BaseModel's context you could use TypeAdapter as such:

validated_grade_history = TypeAdapter(GradeHistory).validate_python(obj_to_validate)

now validated_grade_history is annotated by the IDE as GradeHistory and it's also validated by pydantic.

[innicoder]

I don’t mind this but the problem you don’t seem to be getting is that many of our databases were using root and it was saved like that in the database, which means we somehow have to do db migrations and I don’t know how to do that.

On Wed, 3 Jan 2024 at 10:31, Guy Tsitsiashvili @.***> wrote:

Pydantic V2 introduced TypeAdapter https://docs.pydantic.dev/latest/api/type_adapter/ and annotated types with validators. Using RootModel feels a little anti-pattern in V2, maybe consider working with TypeAdapter more often,

In your example (and given you don't directly access GradeHistory you could define it as such

GradeHistory = List[Grade]

now it has become a type and is still validated in the context of a BaseModel and its descendants (View and Document)

In a case where you need to validate GradeHistory out of a BaseModel's context you could use TypeAdapter as such:

validated_grade_history = TypeAdapter(GradeHistory).validate_python(obj_to_validate)

now validated_grade_history is annotated by the IDE as GradeHistory and it's also validated by pydantic.

— Reply to this email directly, view it on GitHub https://github.com/roman-right/beanie/issues/796#issuecomment-1875072620, or unsubscribe https://github.com/notifications/unsubscribe-auth/AX3T4JJPDFK73CXMSRVM4B3YMUQOVAVCNFSM6AAAAABAFDID5OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZVGA3TENRSGA . You are receiving this because you authored the thread.Message ID: @.***>

[GuyGooL5]

That's seems like a deep problem, implementing an iterative migration might be daunting. I can recommend custom validation for this case. Assuming your data is saved somewhat like this

// This is a representation of a saved grade history in the DB
{
     "__root__": [...]
}

You could write a wrap validator to validate it as a list (using Annotated Validators )

from typing import Any, Annotated
from pydantic import ValidatorFunctionWrapHandler
from pydantic.functional_validators import WrapValidator

def validate_grade_list(v: Any, handler: ValidatorFunctionWrapHandler):
    if isinstance(v, dict):
        return handle(v.get("__root__")) # you can also add further validation
    return handle(v)
    
GradeHistory = Annotated[List[Grade], WrapValidator(validate_grade_list)]

The function validate_grade_list adds a prior step to validating grade-history DB entries to try and validate the inner __root__ property given the entry is a dict, otherwise it forwards the handling to the default pydantic handler which validates it as a list of Grade models.

The annotated part makes it so whenever GradeHistory is used as a type in a BaseModel or in a context of TypeAdapter, the validate_grade_list function will be used to validate instead of the default validator. Regarding serialization, the value should be now serialized simply as a list, not an object with __root__ property.

Note that this method might create a mixed shape in your DB after implementing this approach, but it should work just fine by de-serializing existing entries.

I would however recommend migrating the older entries to a uniform shape.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been stalled for 14 days with no activity.