django-storages icon indicating copy to clipboard operation
django-storages copied to clipboard

CreateMultipartUpload operation: Invalid Request with s3 storage

Open Tudyx opened this issue 2 years ago • 1 comments

I have the error CreateMultipartUpload operation: Invalid Request (request has multiple authentication types, please use one) while uploading a file superior to 5 Mo with django. When the file size is superior to 5 Mo, the boto3 client start to split the file to upload it using multipart upload. The cause of the problem seems to be the content-type of the request of the POST request for creating the CreateMultipartUpload, which is set to multipart/form-data

I used a custom boto3 client and i did't get the error (i used Minio locally).

import boto3

client = session.client(
    service_name="s3",
    aws_access_key_id="foo",
    aws_secret_access_key="foobarfoobar",
    endpoint_url="http://localhost:9000/",
)
client.upload_file(
    "path/big_file.zip, "my-bucket", "bigfile"
)

I checked the requests, the only difference is content-type header which is set to multipart/form-data from django storage and nothing with my boto3 client.

If i set manually the content type of my boto3 client to application/octet-stream (as it's the default in django-storage ) it works also.

The content type seems to only be set in S3Boto3Storage._get_write_parameters function. As the type guessed by mimetypes is application/zip and the self.default_content_type is application/octet-stream, i suppose the multipart/form-data come from getattr(content, 'content_type', None)

For now i solved my problem by setting the content type globally with AWS_S3_OBJECT_PARAMETERS

AWS_S3_OBJECT_PARAMETERS = {
    "CacheControl": f"max-age={_AWS_EXPIRY}, s-maxage={_AWS_EXPIRY}, must-revalidate",
    "ContentType": "application/octet-stream",
}

Please could you indicate me if there is a cleaner way to do it?

Tudyx avatar Jun 02 '22 16:06 Tudyx

I have a similar issue when uploading an Excel file superior to 5 Mo with django to Minio. The file is corrupted and impossible to read back.

Setting the content type as mentioned above didn`t solve the problem.

Any update on this issue?

courcelm avatar Apr 04 '23 17:04 courcelm