dataclasses-json icon indicating copy to clipboard operation
dataclasses-json copied to clipboard

Strange forward reference behavior with Marshmallow schema

Open bpdavis86 opened this issue 2 years ago • 1 comments

If I try to use Marshmallow o.schema().dumps() to serialize vs o.to_json() and o involves nested classes with forward reference, I get an error.

from dataclasses import dataclass
from dataclasses_json import dataclass_json
from typing import List


@dataclass_json
@dataclass
class A:
    bar: 'B'  # forward declared class name
    baz: List['C']  # forward declared in list
    foo: str = ""


@dataclass_json
@dataclass
class B:
    x: int = 0
    y: int = 1


@dataclass_json
@dataclass
class C:
    a: str = ""


if __name__ == '__main__':
    b = B(x=1, y=2)
    c1 = C(a='Paul')
    c2 = C(a='John')
    c3 = C(a='Ringo')
    a = A(foo='Hello', bar=b, baz=[c1, c2, c3])
    # works fine
    s1 = a.to_json()
    # fail with TypeError
    s2 = A.schema().dumps(a, indent=2)

This gives Marshmallow warnings:

C:\Users\...\py39\lib\site-packages\dataclasses_json\mm.py:271: UserWarning: Unknown type B at A.bar: B It's advised to pass the correct marshmallow type to `mm_field`.
  warnings.warn(
C:\Users\...\py39\lib\site-packages\dataclasses_json\mm.py:271: UserWarning: Unknown type ForwardRef('C') at A.baz: typing.List[ForwardRef('C')] It's advised to pass the correct marshmallow type to `mm_field`.
  warnings.warn(

and finally error

TypeError: Object of type C is not JSON serializable

(In this case there is no actual need for forward references, this example is contrived to illustrate the behavior)

I understand an issue related to forward reference is still being tracked in #5. Perhaps this behavior is already known.

bpdavis86 avatar Feb 22 '22 22:02 bpdavis86

I should comment that a workaround for the issue is to defer applying the decorator and manually modify the annotations to remove the forward reference.

from dataclasses import dataclass
from dataclasses_json import dataclass_json
from typing import List

# defer decoration till later
# @dataclass_json
# @dataclass
class A:
    bar: 'B'  # forward declared class name
    baz: List['C']  # forward declared in list
    foo: str = ""


@dataclass_json
@dataclass
class B:
    x: int = 0
    y: int = 1


@dataclass_json
@dataclass
class C:
    a: str = ""


# Fix the problematic forward reference annotations now that we have these types
A.__annotations__['bar'] = B
A.__annotations__['baz'] = List[C]
# Apply the decorators
A = dataclass(A)
A = dataclass_json(A)


if __name__ == '__main__':
    b = B(x=1, y=2)
    c1 = C(a='Paul')
    c2 = C(a='John')
    c3 = C(a='Ringo')
    a = A(foo='Hello', bar=b, baz=[c1, c2, c3])
    # works fine
    s1 = a.to_json()
    # It works now
    s2 = A.schema().dumps(a, indent=2)

bpdavis86 avatar Feb 22 '22 22:02 bpdavis86