dataclasses-json icon indicating copy to clipboard operation
dataclasses-json copied to clipboard

Marshmallow fields in global configuration are ignored when calling .schema()

Open g0di opened this issue 3 years ago • 1 comments

Hello there,

I just discovered that we could register our own default encoders/decoders/marshallow fields per types to affect all json dataclasses. However, it looks likes the mm_fields defined in global_config object are barely ignored when generating schemas on the fly.

The following snippet of code illustrate this

from dataclasses import dataclass
from datetime import datetime
import marshmallow as ma
from dataclasses_json import DataClassJsonMixin, global_config

global_config.decoders.update(
    {
        datetime: datetime.fromisoformat,
    }
)
global_config.mm_fields.update(
    {
        datetime: ma.fields.DateTime(format="iso"),
    }
)


@dataclass
class Foo(DataClassJsonMixin):
    foo: datetime

# This work, the iso formatted date is properly decoded
Foo.from_dict({"foo": "2020-12-01T12:00:00.000+00:00"})
# Got a type error because it uses the default datetime field which try to decode the value as a timestamp (so an integer instead of string)
Foo.schema().load({"foo": "2020-12-01T12:00:00.000+00:00"})

It gives me the following output

Traceback (most recent call last):
  File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/test.py", line 28, in <module>
    Foo.schema().load({"foo": "2020-12-01T12:00:00.000+00:00"})
  File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/.venv/lib/python3.7/site-packages/marshmallow/schema.py", line 728, in load
    data, many=many, partial=partial, unknown=unknown, postprocess=True
  File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/.venv/lib/python3.7/site-packages/marshmallow/schema.py", line 866, in _do_load
    unknown=unknown,
  File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/.venv/lib/python3.7/site-packages/marshmallow/schema.py", line 674, in _deserialize
    index=index,
  File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/.venv/lib/python3.7/site-packages/marshmallow/schema.py", line 496, in _call_and_store
    value = getter_func(data)
  File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/.venv/lib/python3.7/site-packages/marshmallow/schema.py", line 667, in <lambda>
    val, field_name, data, **d_kwargs
  File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/.venv/lib/python3.7/site-packages/marshmallow/fields.py", line 346, in deserialize
    output = self._deserialize(value, attr, data, **kwargs)
  File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/.venv/lib/python3.7/site-packages/dataclasses_json/mm.py", line 41, in _deserialize
    return _timestamp_to_dt_aware(value)
  File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/.venv/lib/python3.7/site-packages/dataclasses_json/utils.py", line 111, in _timestamp_to_dt_aware
    dt = datetime.fromtimestamp(timestamp, tz=tz)
TypeError: an integer is required (got type str)

Thanks for this very interesting library which deserve much more attention

Using dataclasses-json==0.5.2, python==3.7

g0di avatar Dec 04 '20 17:12 g0di

I think I am seeing a similar issue:

import dataclasses
import datetime
import typing

import dataclasses_json
import marshmallow


@dataclasses_json.dataclass_json
@dataclasses.dataclass(frozen=True)
class Example:
    some_datetime_field: typing.Optional[datetime.datetime] = dataclasses.field(
        default=None,
        metadata=dataclasses_json.config(
            encoder=datetime.datetime.isoformat,
            decoder=lambda x: datetime.datetime.fromisoformat(str(x)) if x else None,
            mm_field=marshmallow.fields.DateTime(format='iso'),
        )
    )


serialised = '{"some_datetime_field": null}'
Example.from_json(serialised)  # Works fine: `Example(some_datetime_field=None)`

Example.schema().loads(serialised)  # Uh-oh: `ValidationError: {'some_datetime_field': ['Field may not be null.']}`

Torvaney avatar May 03 '21 22:05 Torvaney

I believe the issue is this line https://github.com/lidatong/dataclasses-json/blob/21d2ab263e833fb48e59ac7b037e1ca59d86b916/dataclasses_json/core.py#L68

this should be global_metadata[field.name]['mm_field'] = mm_fields[field.type]

mtheisen-solea avatar Jun 29 '23 14:06 mtheisen-solea

See pr https://github.com/lidatong/dataclasses-json/pull/253

klaascharlier avatar Jul 20 '23 12:07 klaascharlier