django-ninja icon indicating copy to clipboard operation
django-ninja copied to clipboard

[BUG] PatchDict errors with inherited schemas

Open filippomc opened this issue 1 year ago • 7 comments

Describe the bug

I have a schema hierarchy such as:

class ViewableContent(Schema):
    name: str
    description: str = None

class MySchema(ViewableContent):
   other: str # If I don't add a new field the problem does not arise

Then add a router like the following:

@router.patch('/{uuid}', response={200: MySchema})
@transaction.atomic
def my_update(request: HttpRequest, uuid: str, payload: PatchDict[MySchema]):
   ...

When I run my application the following error is raised:

  File "/home/user/mnp/applications/neuroglass-research/backend/neuroglass_research/api/__init__.py", line 4, in <module>
    from .studies import router as studies_router
  File "/home/user/mnp/applications/neuroglass-research/backend/neuroglass_research/api/studies.py", line 69, in <module>
    def update_study(request: HttpRequest, study_id: int, payload: PatchDict[UpdateStudyPayload]):
                                                                   ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/home/user/miniconda3/envs/mnp/lib/python3.12/site-packages/ninja/patch_dict.py", line 45, in __getitem__
    new_cls = create_patch_schema(schema_cls)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/miniconda3/envs/mnp/lib/python3.12/site-packages/ninja/patch_dict.py", line 29, in create_patch_schema
    t = schema_cls.__annotations__[f]
        ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
KeyError: 'name'

Versions:

  • Python version: 3.12
  • Django version: 5.1.2
  • Django-Ninja version: 1.3.0
  • Pydantic version: 2.9.2

filippomc avatar Oct 22 '24 10:10 filippomc

Possibly related discussion: https://github.com/pydantic/pydantic/discussions/4242

filippomc avatar Oct 22 '24 12:10 filippomc

If I don't add a new field the problem does not arise

so problem only appears when you ADD some field ? or always ?

vitalik avatar Oct 22 '24 16:10 vitalik

If I don't add a field it's like Pydantic is considering the two classes equivalent and what I see is that the annotations are the same from the base class. When adding a field, that field is the only one within the annotations dict.

filippomc avatar Oct 23 '24 10:10 filippomc

well I still do not understand you.. can you show two examples working and non working or something...

vitalik avatar Oct 23 '24 14:10 vitalik

The not working example is the one I originally posted, and the application breaks on any path with the error above. These are a few ones that are working for me:

Working -- no inheritance:

class MySchema(Schema):
   name: str
   description: str = None
   other: str

Working -- duplicate all base class fields

class ViewableContent(Schema):
   name: str
   description: str = None

class MySchema(ViewableContent):
   name: str = None
   description: str = None
   other: str 

Working -- subclassing but no new fields

class ViewableContent(Schema):
   name: str
   description: str = None

class MySchema(ViewableContent):
   pass

Created an example project with all the examples

filippomc avatar Oct 24 '24 08:10 filippomc

Hi

I have encountered the same bug with inherited Schemas

The exception originates from the following method, where it uses the __annotations__ to decide if the field type needs to become Optional :

https://github.com/vitalik/django-ninja/blob/49f284a319b9d657c2fab7f01307c42e0abbd487/ninja/patch_dict.py#L26-L40

In line 29 sometimes it cannot find the root Schema fields, I think is related with how python lazy-create and resolves __annotations__

https://docs.python.org/3/howto/annotations.html#accessing-the-annotations-dict-of-an-object-in-python-3-10-and-newer

If no field is added to leaf Schema/Class no __annotations__ are created and when it is accessed the root __annotations__ is returned with all root Schema Fields.

If a field or more are added to leaf Schema/Class then the created __annotations__ is not an intersection between root a leaf class field, but only leaf Schema fields (among other stuff…), failing to find root Schema fields,.

In my project, I temporarily fix the issue using the suggest approach to get fields types in this pydantic discussion: https://github.com/pydantic/pydantic/discussions/2360

from typing import get_type_hints
//(...)
def fix_create_patch_schema(schema_cls: Type[Any]) -> Type[ModelToDict]:
    values, annotations = {}, {}
    schema_cls_type_hints = get_type_hints(schema_cls)
    for f in schema_cls.__fields__.keys():
        t = schema_cls_type_hints[f]
        if not is_optional_type(t):
            values[f] = getattr(schema_cls, f, None)
            annotations[f] = Optional[t]
    values["__annotations__"] = annotations
    OptionalSchema = type(f"{schema_cls.__name__}Patch", (schema_cls,), values)

    class OptionalDictSchema(ModelToDict):
        _wrapped_model = OptionalSchema
        _wrapped_model_dump_params = {"exclude_unset": True}

    return Body[OptionalDictSchema]

Given my lack of knowledge about the Django-ninja structure.

Do you think is a good solution ? Should I make a PR ?

EduardoCastanho avatar Jan 13 '25 13:01 EduardoCastanho

Any update on this?

iStorry avatar Mar 10 '25 07:03 iStorry