marshmallow icon indicating copy to clipboard operation
marshmallow copied to clipboard

List(Nested()) doesn't use `many` in the nested schema

Open ThiefMaster opened this issue 4 years ago • 3 comments

from marshmallow import Schema, fields, post_dump


class ItemSchema(Schema):
    id = fields.Integer()
    name = fields.String()

    @post_dump(pass_many=True)
    def query_extra_data(self, data, many, **kwargs):
        print(f'post_dump called: {many=} {data=}')


class TestSchema(Schema):
    items = fields.List(fields.Nested(ItemSchema))


data = {'items': [
    {'id': 1, 'name': 'foo'},
    {'id': 2, 'name': 'bar'},
    {'id': 3, 'name': 'snafu'},
]}

TestSchema().dump(data)

This gives me the following output:

post_dump called: many=False data={'name': 'foo', 'id': 1}
post_dump called: many=False data={'name': 'bar', 'id': 2}
post_dump called: many=False data={'name': 'snafu', 'id': 3}

However, I expected this:

post_dump called: many=True data=[{'name': 'foo', 'id': 1}, {'name': 'bar', 'id': 2}, {'name': 'snafu', 'id': 3}]

This is a problem when the post_dump hook queries stuff from a database as I would end up with n individual queries instead of a single one where I can use IN to efficiently get data for all the objects in the list.

Are there any decent workarounds for this? One thing that came to my mind is using this instead of List(Nested), but it feels much uglier...

items = fields.Function(lambda data: ItemSchema(many=True).dump(data['items']))

ThiefMaster avatar Apr 30 '21 09:04 ThiefMaster

I just realized that Nested(..., many=True) is still a thing. I had the impression it was removed in marshmallow 3 but apparently that's not the case, since this still works perfectly fine:

items = fields.Nested(ItemSchema, many=True)

However, if #779 happens this would break.

ThiefMaster avatar Apr 30 '21 09:04 ThiefMaster

Just stumbled upon this. So what is the correct approach for this? Using fields.List or using nested with many=True? 😕

aalvrz avatar Mar 11 '22 18:03 aalvrz

Another poor soul here having the same question. I am about to replace fields.List with Nested(many=True) as I want to reuse a validation without the need to change things in the parent schemas.

@sloria Could you enlighten us if Nested(many=True) will be a thing in the future or not? And if not how to handle the validation case from above?

aberres avatar Oct 11 '22 12:10 aberres