desert icon indicating copy to clipboard operation
desert copied to clipboard

desert crash on TYPE_CHECKING encapsulated type hints and ignores exclude

Open sveinse opened this issue 3 years ago • 1 comments

Using type hints that is only pulled in via if TYPE_CHECKING constructs will fail serialization, even if the field is excluded.

from typing import TYPE_CHECKING, Optional
import desert
import attr

if TYPE_CHECKING:
    from twisted.internet.defer import Deferred

@attr.s
class A:
    a: int = attr.ib()
    b: Optional['Deferred'] = attr.ib(default=None, repr=False)

a = A(1)
s = desert.schema(A, meta={'exclude': ('b', )})
d = s.dump(a)

In this example, Deferred is pulled in under the TYPE_CHECKING paradigm. There might be good reasons to pull a type hint this way, e.g. to avoid circular references. This cause desert._make to crash. The expected outcome would be that since the field is excluded in the schema specification, desert wouldn't need its type hint.

Traceback (most recent call last):
  File "C:\sveinse\desert_bug.py", line 15, in <module>
    s = desert.schema(A, meta={'exclude': ('b', )})
  File "C:\sveinse\venv\lib\site-packages\desert\__init__.py", line 24, in schema
    return desert._make.class_schema(cls, meta=meta)(many=many)
  File "C:\sveinse\venv\lib\site-packages\desert\_make.py", line 127, in class_schema
    hints = t.get_type_hints(clazz)
  File "C:\Users\sveinse\AppData\Local\Programs\Python\Python310\lib\typing.py", line 1808, in get_type_hints
    value = _eval_type(value, base_globals, base_locals)
  File "C:\Users\sveinse\AppData\Local\Programs\Python\Python310\lib\typing.py", line 328, in _eval_type
    ev_args = tuple(_eval_type(a, globalns, localns, recursive_guard) for a in t.__args__)
  File "C:\Users\sveinse\AppData\Local\Programs\Python\Python310\lib\typing.py", line 328, in <genexpr>
    ev_args = tuple(_eval_type(a, globalns, localns, recursive_guard) for a in t.__args__)
  File "C:\Users\sveinse\AppData\Local\Programs\Python\Python310\lib\typing.py", line 326, in _eval_type
    return t._evaluate(globalns, localns, recursive_guard)
  File "C:\Users\sveinse\AppData\Local\Programs\Python\Python310\lib\typing.py", line 691, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>

sveinse avatar May 08 '22 18:05 sveinse

I found a workaround by using a custom do-nothing Field and inject it into the attr class with desert.ib():

import attr
import desert
import marshmallow
from twisted.internet.defer import Deferred

class NullField(marshmallow.fields.Field):
    ''' Do-nothing field '''

    def _serialize(self, value, attr, obj, **kwargs):
        return None

    def _deserialize(self, value, attr, data, **kwargs):
        return None

@attr.s
class A:
    a: int = attr.ib()
    b: Optional[Deferred] = desert.ib(NullField(), default=None, repr=False)

a = A(1)
s = desert.schema(A, meta={'exclude': ('b', )})
d = s.dump(a)

I've realized that one cannot use TYPE_CHECKING scoped objects with desert because it relies on typing.get_type_hints(). This call requires access to the referenced type object. The following example won't work with serialization.

if TYPE_CHECKING:
    from foo import Bar

class A:
    a: 'Bar': attr.ib()   # Will not work

What I think this all boils down to is a better way to exclude fields from arbitrary desert/marshmallow serialization. Is there another way to exclude fields other than meta={'exclude': ('a', 'b')} when creating the schema? E.g. in the field definitions?

sveinse avatar May 08 '22 20:05 sveinse