Trying SSE-C

[ahnkle]

I liked the idea of server side encryption with my keys, so I tried

#!/bin/sh -ex
#
#

key=$(echo -n "key" | base64)
key_md5=$(echo -n "key" | md5sum | cut -b 1-32 | base64)

s3cmd \
  --add-header=x-amz-server-side-encryption-customer-algorithm:AES256 \
  --add-header=x-amz-server-side-encryptiony:"$key" \
  --add-header=x-amz-server-side-encryption-customer-key-MD5:"$key_md5" \
  $@

but I keep getting

ERROR: S3 error: 403 (SignatureDoesNotMatch): The request signature we calculated does not match the signature you provided. Check your key and signing method.

Any ideas?

Example data values:

base64 of "key": a2V5 base64 of md5 of "key": M2M2ZTBiOGE5YzE1MjI0YTgyMjhiOWE5OGNhMTUzMWQK

[arunbhagyanath]

Any idea about this ? I'm also facing the same error.

Version used: s3cmd version 1.5.0-rc1 key: test Syntax I used: ./s3cmd --add-header=x-amz-server-side-encryption-customer-algorithm:AES256 --add-header=x-amz-server-side-encryption-key:"dGVzdA==" --add-header=x-amz-server-side-encryption-customer-key-MD5:"MDk4ZjZiY2Q0NjIxZDM3M2NhZGU0ZTgzMjYyN2I0ZjY=" sync /home/users/arun/Desktop s3://Test_Version

[hrchu]

x-amz-* headers should be added to the request signature.

[ahnkle]

I got a step closer.

The headers are all lower case.

s3cmd \
  --no-check-md5 \
  --add-header=x-amz-server-side-encryption-customer-key:$key \
  --add-header=x-amz-server-side-encryption-customer-key-md5:$key_md5 \
  --add-header=x-amz-server-side-encryption-customer-algorithm:AES256 \
  $@

But now I get

ERROR: S3 error: 400 (InvalidArgument): The calculated MD5 hash of the key did not match the hash that was provided.

So I guess I am getting closer. I am using s3cmd version 1.5.0-beta1: is there anything in rc1 that might help?

[jheller]

I've got a bit closer. Using 1.5.0-rc1.

You need to base64 encode the binary MD5 hash, not its hex digest form. In shell, the way to do this is with openssl. Also the AES256 key must be 256 bits - 32 chars.

secret="12345678901234567890123456789012"
key=$(echo -n $secret | base64)
key_md5=$(echo -n $secret | openssl dgst -md5 -binary | base64)

s3cmd \
  --add-header=x-amz-server-side-encryption-customer-algorithm:AES256 \
  --add-header=x-amz-server-side-encryptiony:"$key" \
  --add-header=x-amz-server-side-encryption-customer-key-MD5:"$key_md5" \
  $@

We still have a problem though. s3cmd expects the returned ETag to by an md5 digest of the object, but it is not with SSE-C. Presumably its the md5 of the encrypted object, but its different to the hash of the file being uploaded.

For PUT operations, s3cmd retried 6 times then says it has failed, although the object is actually uploaded. You can use the same wrapper script to do a GET and it works, but again complains about md5 signature. The use of the --no-check-md5 option doesn't help.

A sync works for the initial transfer (doing 6 PUTs for each file), but I haven't experimented enough yet with what happens with incremental syncs.

[jheller]

I have forked s3cmd and have been working on adding support for SSE-C. Its at a stage now where I think its good to go, but I'd like more testers before a pull request. https://github.com/jheller/s3cmd Se the wiki for some notes.

[mdomsch]

i'd be open to merging your branch. Care to rebase to current master and submit a pull request? Thanks, Matt

[jrnt30]

@jheller Thanks for note about the binary vs. hex mode. I was banging my head against a way for several hours until I stumbled upon this.

[brice-laurencin]

This command should be in the documentation.

key=$(echo -n $secret | base64)
key_md5=$(echo -n $secret | openssl dgst -md5 -binary | base64)

I just spent literal days to find this.