cattrs icon indicating copy to clipboard operation
cattrs copied to clipboard

Validation errors from multiple fields are not in ClassValidationError

Open Lincoln-GR opened this issue 2 months ago • 7 comments

python version: 3.13.7 attrs version: 25.4.0 cattrs version 25.3.0

When I try to structure a dict into a class, I only see a validation error from one field, when I would expect to see multiple.

Example:

import attrs
import cattrs

@attrs.define()
class Foo:
    a: str = attrs.field(validator=[attrs.validators.min_len(3)])
    b: str = attrs.field(validator=[attrs.validators.min_len(3)])

cattrs.structure(
    {"a": "a", "b": "b"},
    Foo,
)

I am expecting to see validation errors from fields a and b, but only see a:

Traceback (most recent call last):
  File "<cattrs generated structure __main__.Foo>", line 16, in structure_Foo
    return __cl(
      **res,
    )
  File "<attrs generated methods __main__.Foo>", line 28, in __init__
    __attr_validator_a(self, __attr_a, self.a)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/attr/_make.py", line 3279, in __call__
    v(inst, attr, value)
    ~^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/attr/validators.py", line 575, in __call__
    raise ValueError(msg)
ValueError: Length of 'a' must be >= 3: 1

During handling of the above exception, another exception occurred:

  + Exception Group Traceback (most recent call last):
  |   File "/home/lincoln/.config/JetBrains/PyCharm2025.2/scratches/scratch_2.py", line 12, in <module>
  |     cattrs.structure(
  |     ~~~~~~~~~~~~~~~~^
  |         {"a": "a", "b": "b"},
  |         ^^^^^^^^^^^^^^^^^^^^^
  |         Foo,
  |         ^^^^
  |     )
  |     ^
  |   File ".venv/lib/python3.13/site-packages/cattrs/converters.py", line 589, in structure
  |     return self._structure_func.dispatch(cl)(obj, cl)
  |            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  |   File "<cattrs generated structure __main__.Foo>", line 19, in structure_Foo
  |     except Exception as exc: raise __c_cve('While structuring ' + 'Foo', [exc], __cl)
  |                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | cattrs.errors.ClassValidationError: While structuring Foo (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "<cattrs generated structure __main__.Foo>", line 16, in structure_Foo
    |     return __cl(
    |       **res,
    |     )
    |   File "<attrs generated methods __main__.Foo>", line 28, in __init__
    |     __attr_validator_a(self, __attr_a, self.a)
    |     ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
    |   File ".venv/lib/python3.13/site-packages/attr/_make.py", line 3279, in __call__
    |     v(inst, attr, value)
    |     ~^^^^^^^^^^^^^^^^^^^
    |   File ".venv/lib/python3.13/site-packages/attr/validators.py", line 575, in __call__
    |     raise ValueError(msg)
    | ValueError: Length of 'a' must be >= 3: 1
    +------------------------------------

Plus when I pass that exception to cattrs.transform_exception the output list is

['invalid value @ $']

which seems lacking on detail of where the validation error happened.

I am expecting to see multiple because the docs say

When structuring a class, cattrs will gather any exceptions on a field-by-field basis and raise them as a cattrs.ClassValidationError, which is a subclass of BaseValidationError.

I am doing something wrong, or this not a feature of cattrs?

Lincoln-GR avatar Oct 23 '25 05:10 Lincoln-GR

Hello!

Cattrs and attrs validation are completely separate, unfortunately. Cattrs looks at your types (fields a and b) and ensures they are strings. Then it instantiates the attrs class as usual, and the attrs validators run.

The attrs validators short-circuit on the first error, and the error raised isn't linked to the field name in a generic way (as far as I know). So cattrs just sees this exception and returns it without any further metadata. Also for completeness note that attrs validators run always (in cattrs and outside of cattrs), whereas cattrs validation only happens when you structure (edge validation).

@hynek I wonder if there's a way to make the attrs validators not short-circuit?

There are some efforts in progress to add value-based (as opposed to type-based) validation for cattrs but nothing mature yet.

Tinche avatar Oct 23 '25 09:10 Tinche

@Tinche Thanks for the explanation!

Indeed changing my example to be attrs-only

import attrs

@attrs.define()
class Foo:
    a: str = attrs.field(validator=[attrs.validators.min_len(3)])
    b: str = attrs.field(validator=[attrs.validators.min_len(3)])

Foo(a="a", b="b")

gives

Traceback (most recent call last):
  File "/home/lincoln/.config/JetBrains/PyCharm2025.2/scratches/scratch_2.py", line 9, in <module>
    Foo(a="a", b="b")
    ~~~^^^^^^^^^^^^^^
  File "<attrs generated methods __main__.Foo>", line 28, in __init__
    __attr_validator_a(self, __attr_a, self.a)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/attr/_make.py", line 3040, in __call__
    v(inst, attr, value)
    ~^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/attr/validators.py", line 537, in __call__
    raise ValueError(msg)
ValueError: Length of 'a' must be >= 3: 1

I can see the validators are short-circuiting.

I had a quick look at the attrs issue tracker, https://github.com/python-attrs/attrs/issues/1421 seems vagely related, but I couldn't see this specific issue.

LincolnPuzey avatar Oct 24 '25 06:10 LincolnPuzey

@hynek I wonder if there's a way to make the attrs validators not short-circuit?

Are you asking for multi-exceptions? Sounds very breaking?

hynek avatar Oct 24 '25 11:10 hynek

Are you asking for multi-exceptions? Sounds very breaking?

I suppose yes, and yes.

Feel free to tell me I'm dreaming

LincolnPuzey avatar Nov 02 '25 12:11 LincolnPuzey

Hmm, my gut feeling with attrs vs cattrs validation is: attrs catches bugs, cattrs catches invalid user data – if it makes any sense.

The attrs validation framework is meant to enforce invariants like sprinkling asserts through your code base so the reporting is not meant to be human friendly or reportable. OTOH cattrs is meant to take untrusted, user-provided data and potentially tell the user what's what.

I know that packages like Pydantic have blurred these lines but I also think that's not a good thing.


Potentially someone could have a look at the validation logic within attrs and look if it could be expanded easily to allow for something like define(group_validation_errros=True), but given that the __init__ is holy and extremely optimized, I doubt there's an easy way.

hynek avatar Nov 09 '25 06:11 hynek

Hmm, my gut feeling with attrs vs cattrs validation is: attrs catches bugs, cattrs catches invalid user data – if it makes any sense. ... OTOH cattrs is meant to take untrusted, user-provided data and potentially tell the user what's what.

Makes sense to me.

To achieve what I want, sounds like cattrs would need to introspect the attrs validators, and run them as part of its structuring, before __init__.

LincolnPuzey avatar Nov 20 '25 08:11 LincolnPuzey

Hm, ok. The solution will probably be in cattrs.

I don't think we want to inspect attrs validators and run them - we cannot avoid running them when instantiating the class, so they will run twice. I think we want to add cattrs validators (constraints) and let users define them.

Tinche avatar Nov 20 '25 09:11 Tinche