s3proxy icon indicating copy to clipboard operation
s3proxy copied to clipboard

Alias blobstore breaks multipart uploads to S3

Open jrtcppv opened this issue 8 months ago • 2 comments

Using the following s3proxy.conf for an AWS S3 bucket:

s3proxy.endpoint=http://0.0.0.0:8085
s3proxy.authorization=none
jclouds.provider=aws-s3
jclouds.identity=####################
jclouds.credential=####################################
jclouds.region=us-east-2

I am able to complete a multipart upload successfully with aws s3 cp ~/big_file.mp4 s3://test-bucket --endpoint-url http://localhost:8085

However, when I enable alias-blobstore and bucket-locator:

s3proxy.endpoint=http://0.0.0.0:8085
s3proxy.authorization=none
jclouds.provider=aws-s3
jclouds.identity=####################
jclouds.credential=####################################
jclouds.region=us-east-2
s3proxy.alias-blobstore.project-1=test-bucket
s3proxy.bucket-locator.1=project-1

And attempt to upload with aws s3 cp ~/big_file.mp4 s3://project-1 --endpoint-url http://localhost:8085 I receive the following error after uploading seemingly all parts:

upload failed: ../../big_file.mp4 to s3://project-1/big_file.mp4 An error occurred (NoSuchKey) when calling the CompleteMultipartUpload operation: The specified key does not exist.

Not sure if this is relevant but I am invoking s3proxy using the docker container with the latest tag, like so:

docker run --rm -v ~/s3proxy.conf:/s3proxy.conf --publish 8085:8085 --entrypoint /bin/bash andrewgaul/s3proxy -c "java -jar /opt/s3proxy/s3proxy --properties /s3proxy.conf"

jrtcppv avatar Jun 19 '24 04:06 jrtcppv