pydantic

pydantic copied to clipboard

Initial Checks

• [X] I confirm that I'm using Pydantic V2

Description

When a field has a custom validator that returns None, exclude_none seems to 'skip excluding' that field. The same happens with exclude_defaults when the custom serializer returns the default value.

Maybe this is the expected behavior, but in that case it would be nice to note this somewhere, maybe on the model_dumps documentation.

Example Code

from pydantic import BaseModel, field_serializer

class A(BaseModel):
    a: str
    
    @field_serializer('a')
    def serialize_a(self, v, _info):
        return None

a = A(a='abc')
a.model_dump(exclude_none=True)  # {'a': None}

Python, Pydantic & OS Version

pydantic version: 2.1.1
        pydantic-core version: 2.4.0
          pydantic-core build: profile=release pgo=false mimalloc=true
                 install path: /<path>/env/lib/python3.8/site-packages/pydantic
               python version: 3.8.16 (default, Dec  7 2022, 01:27:54)  [Clang 14.0.0 (clang-1400.0.29.202)]
                     platform: macOS-13.3.1-arm64-arm-64bit
     optional deps. installed: ['typing-extensions']

Selected Assignee: @hramezani

I would say it's "how it is", rather than clearly a bug or an intended behaviour.

As per this code, we do those checks before custom logic to serialize the field.

In most cases this is the same and will be faster than serialising, them deciding whether to exclude the field.

I'm open to changing it, but the question is whether this would be a breaking change?

One other option is to raise the PydanticOmit error in your serialize to manually exclude that field.

When we decide to work on this, see this issue https://github.com/pydantic/pydantic/issues/8384 for the case of a model_serializer.

One other option is to raise the PydanticOmit error in your serialize to manually exclude that field.

Would you be able to provide an example of this in action? When I attempt to do that myself, the exception just seems to propagate to the calling code. E.g. using the example from the issue description:

class A(BaseModel):
    a: str

    @field_serializer('a')
    def serialize_a(self, v, _info):
        raise PydanticOmit

a = A(a='abc')
a.model_dump(exclude_none=True)
# Output:
#     return self.__pydantic_serializer__.to_python(
#            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# pydantic_core._pydantic_core.PydanticSerializationError: Error calling function # `serialize_a`: PydanticOmit: PydanticOmit()

I've been wanting the ability to raise a PydanticOmit in a serializer, another discussion that also suggests that could be a solution for custom logic for partial models is here (https://github.com/pydantic/pydantic/discussions/5461#discussioncomment-7411977), as a comment from adriangb. I stopped my research after both looking at that and attempting to do it and seeing the PydanticSerializationError.

As per the discussion, it's apparently known that currently PydanticOmit is not usable in a serializer, only in a validator.

But when I saw this comment above:

One other option is to raise the PydanticOmit error in your serialize to manually exclude that field.

I am now wondering once more if there is a way to use it in a serializer in some way, I would love to know. I have a use case that could use it.

I think this is a related issue so instead of raising a new issue I'm just posting some example code here.

"""Pydantic 2.8.2: field_serializer conflicts with model_dump exclude_defaults"""

from datetime import date

from pydantic import BaseModel, field_serializer
from pydantic_core import PydanticOmit


class Model(BaseModel):
    expiry: date = date.max

    @field_serializer("*")
    def serializer(self, value, _info):
        return value.isoformat() if isinstance(value, date) else value


model = Model(expiry=date.max)
print(model)  # expiry=datetime.date(9999, 12, 31)
data = model.model_dump(mode="json", exclude_defaults=True)
print(data)  # {'expiry': '9999-12-31'}

• I was expecting exclude_defaults to prevent access to the serializer because the field has the default value before model_dump is run
• Instead it seems the presence of the field_serializer means exclude_defaults doesn't operate either before or after serialisation
• Remove field_serializer and the expected output of {} results
• I didn't know about PydanticOmit - it would resolve this issue but it requires specifying the omit twice - once to model_dump and once in the serializer
• There are cases where I'd like to get the field name that's being processed currently
• I also get pydantic_core._pydantic_core.PydanticSerializationError: Error calling function 'serializer': PydanticOmit: PydanticOmit() when trying to use PydanticOmit