Inconsistent included_data contents if the same resource is referenced in different relationships (when requested include_data differs between those relationships)

[Battlesheepu]

Hello! This is nothing too serious (and as long as you're sure the same resource is referenced in different relationships, it's not that hard to have a workaround), but I've figured I still should report it here.

The setup

from marshmallow_jsonapi.schema import Schema
from marshmallow_jsonapi import fields


class WebsiteSchema(Schema):
    id = fields.Str()

    class Meta:
        type_ = 'website'


class AuthorSchema(Schema):
    id = fields.Str()
    website = fields.Relationship(schema=WebsiteSchema, type_='website')

    class Meta:
        type_ = 'author'


class ProjectSchema(Schema):
    id = fields.Str()
    current_maintainer = fields.Relationship(schema=AuthorSchema, type_='author')
    author = fields.Relationship(schema=AuthorSchema, type_='author')

    class Meta:
        type_ = 'project'


website_data = {'id': '1'}
author_data = {'id': '1', 'website': website_data}
project_data = {'id': '1', 'author': author_data, 'current_maintainer': author_data}


schema = ProjectSchema(include_data=('current_maintainer', 'author.website'))

serialized_data = schema.dump(project_data)

print(serialized_data['included'])

The problem

As you can see, the same resource (of type 'author') is in this particular case referenced in both of the fields. The contents of 'included' can be one of the following

[
{'type': 'author', 'id': '1', 'relationships': {'website': {'data': {'type': 'website', 'id': '1'}}}},
{'type': 'website', 'id': '1'}
]

or

[
{'type': 'author', 'id': '1'}, 
{'type': 'website', 'id': '1'}
]

I haven't debugged this much, but it seems that, since it's the same resource in the 'included', the Marshaller has no ability to decide between the current_maintainer and author.website, so the choice loses its consistency.

Possible workaround If you know for sure that both relationships will actually contain the very same resource, the whole things gets easy. For include_data, use only the resource for which you need the deepest nesting level. Here, it would become:

schema = ProjectSchema(include_data=('author.website', ))

If this seems too much of a hassle (or a bug too trivial for any consideration), feel free to let me know.

[etiennecaldichouryatnapta]

Hi, Seems like I went through the same issue! Will try to debug it but will be happy if one of the maintainers has some tips for me. :D

[etiennecaldichouryatnapta]

Ok seems like the issue is coming from

self.root.included_data[(item["type"], item["id"])] = item in marshmallow-jsonapi/marshmallow_jsonapi/fields.py:286

We should merge the new item with included_data already present. Something like:

self.root.included_data[(item["type"], item["id"])] = deep_merge(
    self.root.included_data.get((item["type"], item["id"]), {}),
    item
)

What do you think?

[etiennecaldichouryatnapta]

There is another place which needs modification Here is the modified version:

    def _serialize_included(self, value):
        result = self.schema.dump(value)
        if _MARSHMALLOW_VERSION_INFO[0] < 3:
            data = result.data
        else:
            data = result
        item = data["data"]

        if (item["type"], item["id"]) in self.root.included_data:
            merge(item, self.root.included_data[(item["type"], item["id"])])
        else:
            self.root.included_data[(item["type"], item["id"])] = item

        for key, value in self.schema.included_data.items():
            if key in self.root.included_data: 
                merge(value, self.root.included_data[key])
            else: 
                self.root.included_data[key] = value