marshmallow icon indicating copy to clipboard operation
marshmallow copied to clipboard

Different order of pre_load hooks for List(Nested) and Nested(many=True) fields

Open shashkin opened this issue 3 years ago • 1 comments

Hello, I found a strange difference in load behavior between List(Nested) and Nested(many=True) Consider the following example:

from copy import deepcopy
from marshmallow import Schema, fields, pre_load

class InnerSchema(Schema):
    value = fields.String()

    @pre_load(pass_many=False)
    def add_prefix(self, data, **_):
        print('pre_load inner')
        data = deepcopy(data)
        data['value'] = '_'.join([self.context.get('prefix'), data['value']])
        return data

class MiddleSchema(Schema):
    prefix = fields.String()
    inner = fields.Nested(InnerSchema)

    @pre_load(pass_many=False)
    def store_prefix(self, data, **_):
        print('pre_load middle')
        self.context['prefix'] = data['prefix']
        return data

class ListNestedSchema(Schema):
    data = fields.List(fields.Nested(MiddleSchema))

class NestedManySchema(Schema):
    data = fields.Nested(MiddleSchema, many=True)

obj = {'data': [
    {'prefix': 'foo', 'inner': {'value': 'x'}},
    {'prefix': 'bar', 'inner': {'value': 'z'}}
]}

In this synthetic example I have a list of objects, and I need each of them to pass its own piece if data to nested ones on loading. I'm using context to pass this piece of data to nested schema and expecting each of InnerSchema to receive its own piece of data corresponding to exact parent object. Loading data with ListNestedSchema does exactly what I expect, however NestedManySchema works in a different manner. It looks like the reason is that the order of pre_load hooks execution differs and I'm not sure if it was designed in such way or is it a bug.

>>> ListNestedSchema().load(obj)
pre_load middle
pre_load inner
pre_load middle
pre_load inner
{'data': [{'inner': {'value': 'foo_x'}, 'prefix': 'foo'}, {'inner': {'value': 'bar_z'}, 'prefix': 'bar'}]}


>>> NestedManySchema().load(obj)
pre_load middle
pre_load middle
pre_load inner
pre_load inner
{'data': [{'inner': {'value': 'bar_x'}, 'prefix': 'foo'}, {'inner': {'value': 'bar_z'}, 'prefix': 'bar'}]}

I'm using marshmallow==3.14.1 with Python 3.8.2

shashkin avatar Feb 02 '22 12:02 shashkin

I guess this difference is worth to notice in #779 In my project I had to use exactly List(Nested(...)) field, because I use the same context key for storing each prefix, but I guess if I had an option to access the parent object, like in #940 , I could store all prefixes in context separately and get the exact one in InnerSchema based on which parent object is passed. In that case I could be using either of List(Nested(...)) or Nested(..., many=True) fields

shashkin avatar Feb 02 '22 12:02 shashkin