django-storages
django-storages copied to clipboard
CreateMultipartUpload operation: Invalid Request with s3 storage
I have the error CreateMultipartUpload operation: Invalid Request (request has multiple authentication types, please use one) while uploading a file superior to 5 Mo with django.
When the file size is superior to 5 Mo, the boto3 client start to split the file to upload it using multipart upload.
The cause of the problem seems to be the content-type of the request of the POST request for creating the CreateMultipartUpload, which is set to multipart/form-data
I used a custom boto3 client and i did't get the error (i used Minio locally).
import boto3
client = session.client(
service_name="s3",
aws_access_key_id="foo",
aws_secret_access_key="foobarfoobar",
endpoint_url="http://localhost:9000/",
)
client.upload_file(
"path/big_file.zip, "my-bucket", "bigfile"
)
I checked the requests, the only difference is content-type header which is set to multipart/form-data from django storage and nothing with my boto3 client.
If i set manually the content type of my
boto3client toapplication/octet-stream(as it's the default indjango-storage) it works also.
The content type seems to only be set in S3Boto3Storage._get_write_parameters function.
As the type guessed by mimetypes is application/zip and the self.default_content_type is application/octet-stream, i suppose the multipart/form-data come from getattr(content, 'content_type', None)
For now i solved my problem by setting the content type globally with AWS_S3_OBJECT_PARAMETERS
AWS_S3_OBJECT_PARAMETERS = {
"CacheControl": f"max-age={_AWS_EXPIRY}, s-maxage={_AWS_EXPIRY}, must-revalidate",
"ContentType": "application/octet-stream",
}
Please could you indicate me if there is a cleaner way to do it?
I have a similar issue when uploading an Excel file superior to 5 Mo with django to Minio. The file is corrupted and impossible to read back.
Setting the content type as mentioned above didn`t solve the problem.
Any update on this issue?