Issues when a `attrs` converter is defined on a sequence

[bushkov]

• cattrs version: 1.8.0
• Python version: 3.8.11
• Operating System: macOS 11.5.2

Description

I am trying to use prefer_attrib_converters flag to avoid the fallback of invoking converters registered on attributes with attrib which have a List[T] type. Like in the following which works fine for simple cases:

def to_list(input: Union[List[int], int]) -> List[int]:
    if isinstance(input, int):
        return [input]
    else:
        return input

@attr.s
class A:
    x: List[int] = attr.ib(converter=to_list)


def test_a():
    assert cattr.GenConverter(prefer_attrib_converters=True).structure({"x": 1}, A) == A(x=[1])

However, it doesn't seem to work in case of composition of attrs classes.

What I Did

The following code:

@attr.s
class A:
    a: int = attr.ib()

def to_list(input: Union[List[A], A]) -> List[A]:
    if isinstance(input, A):
        return [input]
    else:
        return input

@attr.s
class B:
    b: List[A] = attr.ib(converter=to_list)


def test_a():
    assert cattr.GenConverter(prefer_attrib_converters=True).structure({"b": {"a": 1}}, B) == B(b=[A(a=1)])

results in the following error:

AssertionError: assert B(b={'a': 1}) == B(b=[A(a=1)])

If I set prefer_attrib_converters to False, I get another error:

in test_a
    assert cattr.GenConverter().structure({"b": {"a": 1}}, B) == B(b=[A(a=1)])
cattr/converters.py:294: in structure
    return self._structure_func.dispatch(cl)(obj, cl)
<cattrs generated structure converter_test.B>:3: in structure_B
    'b': structure_b(o['b'], type_b),
cattr/converters.py:472: in _structure_list
    return [
cattr/converters.py:473: in <listcomp>
    self._structure_func.dispatch(elem_type)(e, elem_type)
<cattrs generated structure converter_test.A>:3: in structure_A
    'a': structure_a(o['a'], type_a),
E   TypeError: string indices must be integers

[Tinche]

Hello!

The issue is prefer_attrib_converters only works with leaf nodes (so, no generics, and your A.x field is typed as a list). The reason for this is what otherwise, a very common use case would break:

@define
class B:
    ...

@define
class A:
    a: Sequence[B] = field(converter=tuple)

I can think of a workaround: normalize your data in a pass before calling converter.structure.

I might have an idea for how to support your use case in cattrs though. Let me think about it.

[bushkov]

Thank you!

So far, the workaround which I use is to introduce a special type alias that incorporates both A and List[A] and register a structure hook on it to do the normalization of data d:

@attr.s
class A:
    a: int = attr.ib()


TypeA = Union[A, List[A]]


@attr.s
class B:
    b: TypeA = attr.ib() # type: List[A]


converter = cattr.GenConverter()


def to_list_hook(d, t):
    if type(d) == list:
        items = d
    else:
        items = [d]
    return converter.structure(items, List[A])


converter.register_structure_hook(TypeA, to_list_hook)


def test_a():
    assert converter.structure({"b": {"a": 1}}, B) == B(b=[A(a=1)])

However this has downsides:

• without # type: List[A] comment in the definition of class B MyPy checks on class B fail since any code that uses instances of class B assumes that B.b is always normalized to list
• in the cases when I want to instantiate B with using A and not List[A], it can be done only through cattrs, B(A(a=1)) fails. This can be avoided by adding back the converter to B, but then we do the conversion twice - when unstructuring using cattrs and later when executing converter of attrs.

[Tinche]

Yeah, this approach seems very messy.

I was thinking of allowing the user to override the structuring hook on a per-attribute basis.

So for your case, you'd have to do the following (roughly):

from attr import define
from cattr import override
from cattr.gen import make_dict_structure_fn

@define
class A:
    a: int

@define
class B:
    b: list[A]

def to_list_hook(d, t):
    return d if isinstance(d) else [d]

converter = GenConverter()
converter.register_structure_hook(B, make_dict_structure_fn(B, converter, b=override(structure_hook=to_list_hook)))

[Tinche]

Oh this was actually added in 23.1.0, so closing this.