mashumaro icon indicating copy to clipboard operation
mashumaro copied to clipboard

Raise the inner exception when parsing nested dataclasses

Open mishamsk opened this issue 11 months ago • 2 comments

Is your feature request related to a problem? Please describe. First, this has been discussed in #83 and dismissed. But I'd like to raise the question and propose a solution yet again.

For the context, which is the same as in #135 - I am working with deeply nested structures, millions of them. For some context - some of the serialized trees that I have are 300MB in msgpack! (that's >1GB in memory after deserialization). And they have ONE root.

So when something breaks during deserialization, the current basic functionality is unusable. Even if I could have inspected the 300MB worth of text, finding what caused an error is impossible. The outer context is not helpful, on the contrary it is detrimental. A typical example - I've changed some model and trying to code a migration from the old one.

They way I've fixed it is by using an unwrapped helper: code and doc

It is almost ideal, but there is one thing that can't be done outside of mashumaro - namely, I can't get the index of child if it is part of a sequence. Only the field name. So for model like this (omitting data class and base class for brevity):

class Child;
    attr: int

class Parent;
    children: tuple[Parent | Child, ...]

if I say pass a string that is not an integer in a leaf Child node 10 levels deep, my helper will tell me the path children.children.[8 times more].attr with mashumaro's inner exception. But I won't know the indexes along the way.

Describe the solution you'd like At least for the InvalidFieldValue exception add path capturing, including the index of item in a sequence.

And then either:

  • make what my helper does the new behavior
  • make it a config option to let InvalidFieldValue behave in such a way for backward compatibility

Describe alternatives you've considered See above

Additional context I can try my hand at this if the concept seems right. Didn't want to do a PR before we discuss this

mishamsk avatar Aug 03 '23 04:08 mishamsk