add nested multipart support

[thommignot]

This pull request is a answer to the need to test nested file uploads.

However, a question may be discussed: is Django-Rest-Framework not supporting nested multipart rendering by choice, for some technical reasons, or maybe just because the need has not been felt by any of its contributors?

So I'm suggesting so solution I use for a personnal project that works well in encoding nested lists and dicts and dicts of lists or dicts of lists of dicts :smile:

About that, another question should be asked:

• should I fix a maximum recursion level ? (do serializers have that kind of limit?)

Technically then, its goal is to transcribe complex objects to a form that can be parsed by rest_framework.utils.html.parse_html_list() and rest_framework.utils.html.parse_html_dict(). Thus for example: {'foo': 42, 'bar': [{'django': 'rest', 'framework': 'is'}, {'so': {'great': 'thx'}}]} gives us thoses keys: name="foo" name="bar[0]django" name="bar[0]framework" name="bar[1]so.great"

Given the way parsing is done, the ability to nest serializers, and the generic serializers already integrated in Django-Rest-Framework, it was a pleasure to add that little renderer, so really, thank for your efforts all that time.

Thomas Mignot.

[imomaliev]

This seems to be a pretty standalone feature. Why not just put it into a separate package and distribute it via pip?

[thommignot]

because It is usefull only when testing a REST API, which itself allows nesting serializers but there was no way to test such a feature with multipart/form-data encoding with the current APIClient. Let say you have nested files, you simply cannot test your endpoint except by making a long object such as:

payload = {
    'level0.level2[0]level3_a': item,
    'level0.level2[0]level3_b': item2,
    'level0.level2[1]level3_a': item3,
    'level0.level2[1]level3_b': item4,
...

to obtain a complex object with files in it.

So it WILL be parse, but it's not very straightforward when writing your tests.

[imomaliev]

Still, there is no actual reason to ship it with django-rest-framework itself. It could be installed for testing reasons as separate python package.

[thommignot]

I think that as DRF support this format from the API point, one should be able to test it, thus it seems that DRF should support testing to me. All the point might be there: is this feature missing, can it be usefull? May be there is an other and better way to avoid the problem I'm explaining here or perhaps we should open on issue about not being able to post nested files? DRF supports nesting through JSON, but JSON isn't able to encode file, multipart is. But multipart (in the actual implementation) cannot encode nested data. This is for sure a limitation as it's just a limitation of the encoder.

Maybe we should just rewrite the actual encoder to allow it? In this case yeah, I'd PR django itself. But why? it's not an algorithm from django that parses the result, it is a rest_framework's one. Moreover it is usefull because of DRF and its serializers.

We might think that not everyone will need it. But hey, who does use ALL the feature a lib that usefull as DRF has.

EDIT: I just read https://www.django-rest-framework.org/community/third-party-packages/#about-third-party-packages I understand better now what you were telling me

[Genarito]

Hi @thommignot! This is an excellent job! I had the same problem yesterday: I had nested structures (JSON was working fine), but a file was added. If I changed the form to multipart the File worked but the nested didn't, if I keep JSON no File could be sent. I end up using drf-extra-fields and sending the file as Base64 but I had to make a lot of changes instead of keeping it all simple.

Did you upload a Third-Party library? That would be very helpful!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[auvipy]

Long ago I faced uploding multiple files / nested multipart files uploads as well. so we need to make sure supporting that is aligned with the frameworks goal. but I think that should be possible by default

[tomchristie]

I think that should be possible by default

I don't think we should be adding new functionality to REST framework at this point in time. It's super extensible already, and new functionality can be in the form of third party packages.

[auvipy]

I think that should be possible by default

I don't think we should be adding new functionality to REST framework at this point in time. It's super extensible already, and new functionality can be in the form of third party packages.

I will consider this for an extension I manage then https://github.com/chibisov/drf-extensions

[tomchristie]

Appreciate the input, tho... https://github.com/encode/django-rest-framework/blob/master/CONTRIBUTING.md

At this point in its lifespan we consider Django REST framework to be essentially feature-complete. We may accept pull requests that track the continued development of Django versions, but would prefer not to accept new features or code formatting changes.