dataclasses-json
dataclasses-json copied to clipboard
Marshmallow fields in global configuration are ignored when calling .schema()
Hello there,
I just discovered that we could register our own default encoders/decoders/marshallow fields per types to affect all json dataclasses. However, it looks likes the mm_fields
defined in global_config
object are barely ignored when generating schemas on the fly.
The following snippet of code illustrate this
from dataclasses import dataclass
from datetime import datetime
import marshmallow as ma
from dataclasses_json import DataClassJsonMixin, global_config
global_config.decoders.update(
{
datetime: datetime.fromisoformat,
}
)
global_config.mm_fields.update(
{
datetime: ma.fields.DateTime(format="iso"),
}
)
@dataclass
class Foo(DataClassJsonMixin):
foo: datetime
# This work, the iso formatted date is properly decoded
Foo.from_dict({"foo": "2020-12-01T12:00:00.000+00:00"})
# Got a type error because it uses the default datetime field which try to decode the value as a timestamp (so an integer instead of string)
Foo.schema().load({"foo": "2020-12-01T12:00:00.000+00:00"})
It gives me the following output
Traceback (most recent call last):
File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/test.py", line 28, in <module>
Foo.schema().load({"foo": "2020-12-01T12:00:00.000+00:00"})
File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/.venv/lib/python3.7/site-packages/marshmallow/schema.py", line 728, in load
data, many=many, partial=partial, unknown=unknown, postprocess=True
File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/.venv/lib/python3.7/site-packages/marshmallow/schema.py", line 866, in _do_load
unknown=unknown,
File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/.venv/lib/python3.7/site-packages/marshmallow/schema.py", line 674, in _deserialize
index=index,
File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/.venv/lib/python3.7/site-packages/marshmallow/schema.py", line 496, in _call_and_store
value = getter_func(data)
File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/.venv/lib/python3.7/site-packages/marshmallow/schema.py", line 667, in <lambda>
val, field_name, data, **d_kwargs
File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/.venv/lib/python3.7/site-packages/marshmallow/fields.py", line 346, in deserialize
output = self._deserialize(value, attr, data, **kwargs)
File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/.venv/lib/python3.7/site-packages/dataclasses_json/mm.py", line 41, in _deserialize
return _timestamp_to_dt_aware(value)
File "/Users/gody/dev/earthlab/workspace/internal/workers/pipelines-build-worker/.venv/lib/python3.7/site-packages/dataclasses_json/utils.py", line 111, in _timestamp_to_dt_aware
dt = datetime.fromtimestamp(timestamp, tz=tz)
TypeError: an integer is required (got type str)
Thanks for this very interesting library which deserve much more attention
Using dataclasses-json==0.5.2, python==3.7
I think I am seeing a similar issue:
import dataclasses
import datetime
import typing
import dataclasses_json
import marshmallow
@dataclasses_json.dataclass_json
@dataclasses.dataclass(frozen=True)
class Example:
some_datetime_field: typing.Optional[datetime.datetime] = dataclasses.field(
default=None,
metadata=dataclasses_json.config(
encoder=datetime.datetime.isoformat,
decoder=lambda x: datetime.datetime.fromisoformat(str(x)) if x else None,
mm_field=marshmallow.fields.DateTime(format='iso'),
)
)
serialised = '{"some_datetime_field": null}'
Example.from_json(serialised) # Works fine: `Example(some_datetime_field=None)`
Example.schema().loads(serialised) # Uh-oh: `ValidationError: {'some_datetime_field': ['Field may not be null.']}`
I believe the issue is this line https://github.com/lidatong/dataclasses-json/blob/21d2ab263e833fb48e59ac7b037e1ca59d86b916/dataclasses_json/core.py#L68
this should be global_metadata[field.name]['mm_field'] = mm_fields[field.type]
See pr https://github.com/lidatong/dataclasses-json/pull/253