Custom field that inherits from DictField results in strange behaviour when calling .from_json

[mcleantom]

I want to save pydantic models in my mongo database, so I made a custom field that inherits from DictField:

class BaseModelField(DictField):
    """
    A pydantic BaseModel field. Stores in the database using pydantic.BaseModel.dict().
    """

    def __init__(self, basemodel: Type[BaseModel], **kwargs):
        self._basemodel = basemodel
        super().__init__(**kwargs)

    def validate(self, value, *args, **kwargs):
        # Check correct type of basemodel
        if not isinstance(value, self._basemodel):
            self.error(f"Must be type {type(self._basemodel)}, not {type(value)}")
        # Further validation
        super().validate(value.dict())

    def to_mongo(self, value, *args, **kwargs):
        if isinstance(value, BaseModel):
            return super().to_mongo(value.dict(), *args, **kwargs)
        return super().to_mongo(value, *args, **kwargs)

    def to_python(self, value):
        if issubclass(type(value), self._basemodel):
            return value
        if isinstance(value, str):
            value = self._basemodel.parse_raw(value)
        if isinstance(value, dict):
            value = self._basemodel(**value)
        return value

When I make a document with this field, it can serialize to JSON fine, but when it de-serializes from json, it converts the basemodel field into a list of tuples.

from mongoengine.fields import DictField
from mongoengine.document import Document, EmbeddedDocument
from pydantic import BaseModel
from typing import Type

# Connect do db...


class BM(BaseModel):
    name: str


class Test(Document):
    data = BaseModelField(basemodel=BM)


t = Test(data=BM(name="Tom"))

The first time I call .to_json It works fine:

t.to_json()

Outputs:
'{"data": {"name": "Tom"}}'

However, If I call .from_json, It converts the dictionary to a list

rtn = Test.from_json(t.to_json())
print(rtn.data)

Outputs
[('name', 'Tom')]

And if I now get the .data attribute on the instance t, it has been modified:

t.data
[('name', 'Tom')]

I cannot work out what is happening. How can I serialize a pydantic model in this way into and from a mongo database?