s3proxy
s3proxy copied to clipboard
Alias blobstore breaks multipart uploads to S3
Using the following s3proxy.conf
for an AWS S3 bucket:
s3proxy.endpoint=http://0.0.0.0:8085
s3proxy.authorization=none
jclouds.provider=aws-s3
jclouds.identity=####################
jclouds.credential=####################################
jclouds.region=us-east-2
I am able to complete a multipart upload successfully with aws s3 cp ~/big_file.mp4 s3://test-bucket --endpoint-url http://localhost:8085
However, when I enable alias-blobstore
and bucket-locator
:
s3proxy.endpoint=http://0.0.0.0:8085
s3proxy.authorization=none
jclouds.provider=aws-s3
jclouds.identity=####################
jclouds.credential=####################################
jclouds.region=us-east-2
s3proxy.alias-blobstore.project-1=test-bucket
s3proxy.bucket-locator.1=project-1
And attempt to upload with aws s3 cp ~/big_file.mp4 s3://project-1 --endpoint-url http://localhost:8085
I receive the following error after uploading seemingly all parts:
upload failed: ../../big_file.mp4 to s3://project-1/big_file.mp4 An error occurred (NoSuchKey) when calling the CompleteMultipartUpload operation: The specified key does not exist.
Not sure if this is relevant but I am invoking s3proxy using the docker container with the latest
tag, like so:
docker run --rm -v ~/s3proxy.conf:/s3proxy.conf --publish 8085:8085 --entrypoint /bin/bash andrewgaul/s3proxy -c "java -jar /opt/s3proxy/s3proxy --properties /s3proxy.conf"