django-storages
django-storages copied to clipboard
CreateMultipartUpload operation: Invalid Request with s3 storage
I have the error CreateMultipartUpload operation: Invalid Request (request has multiple authentication types, please use one)
while uploading a file superior to 5 Mo with django.
When the file size is superior to 5 Mo, the boto3
client start to split the file to upload it using multipart upload.
The cause of the problem seems to be the content-type
of the request of the POST request for creating the CreateMultipartUpload
, which is set to multipart/form-data
I used a custom boto3
client and i did't get the error (i used Minio locally).
import boto3
client = session.client(
service_name="s3",
aws_access_key_id="foo",
aws_secret_access_key="foobarfoobar",
endpoint_url="http://localhost:9000/",
)
client.upload_file(
"path/big_file.zip, "my-bucket", "bigfile"
)
I checked the requests, the only difference is content-type
header which is set to multipart/form-data
from django storage
and nothing with my boto3
client.
If i set manually the content type of my
boto3
client toapplication/octet-stream
(as it's the default indjango-storage
) it works also.
The content type seems to only be set in S3Boto3Storage._get_write_parameters
function.
As the type guessed by mimetypes
is application/zip
and the self.default_content_type
is application/octet-stream
, i suppose the multipart/form-data
come from getattr(content, 'content_type', None)
For now i solved my problem by setting the content type globally with AWS_S3_OBJECT_PARAMETERS
AWS_S3_OBJECT_PARAMETERS = {
"CacheControl": f"max-age={_AWS_EXPIRY}, s-maxage={_AWS_EXPIRY}, must-revalidate",
"ContentType": "application/octet-stream",
}
Please could you indicate me if there is a cleaner way to do it?
I have a similar issue when uploading an Excel file superior to 5 Mo with django to Minio. The file is corrupted and impossible to read back.
Setting the content type as mentioned above didn`t solve the problem.
Any update on this issue?