fake-s3 Uploaded chunks using authorization header are not unwrapped before storing

Uploaded chunks using authorization header are not unwrapped before storing

Open cesmoak opened this issue 9 years ago • 7 comments

When performing a chunked upload using the Authorization header, chunks are wrapped with a chunk length and signature as described here: http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html

When storing these chunks, the wrapper needs to be removed before writing chunks. Right now, fake-s3 does not do this, resulting in the wrappers being written to files, corrupting them.

I have a hacky temporary fix made here: https://github.com/cesmoak/fake-s3. I don't claim to know enough about s3 or fake-s3 to say that this is robust, however it seems to work for me.

Oct 05 '15 05:10 cesmoak

We also have that problem.

Nov 23 '15 06:11 spr-build

Have this problem too. Example http://take.ms/R1oEp

There is an issue https://github.com/jubos/fake-s3/issues/126 that attempted resolution but doesn't work for me.

Dec 02 '15 05:12 robzolkos

A work around, depending on your application and test case, is to configure your client to not use chunked uploads:

AmazonS3Client client = new AmazonS3Client(credentials); client.setEndpoint("http://localhost:1234"); client.setS3ClientOptions(S3ClientOptions.builder().setPathStyleAccess(true).disableChunkedEncoding().build());

It would be good to see a fix merged in, but until then hopefully this will save people time in finding a solution.