bji
Can't upload file into Oracle Cloud Storage thru their s3 API
I'm planning to access Oracle Cloud Storage thru the S3 API described here: https://docs.oracle.com/en/cloud/iaas/storage-cloud/cssto/using-s3-api-compatible-clients-access-oracle-storage-cloud-service.html
I used curl to retrieve my S3 secret key (MySecretKey).
I'm using libs3 on the client side and test with s3 binary.
I set the env variables as follow:
export S3_ACCESS_KEY_ID=Storage-acme
export S3_SECRET_ACCESS_KEY=MySecretKey
export S3_HOSTNAME=acme.storage.oraclecloud.com
Where acme is my personal accound and MySecretKey has been authorize as decribe in the doc.
I can list the content of a bucket: ./s3 list clibucket I can download objects from it: ./s3 get clibucket/Vol1/test filename=~/test
BUT I consistantly can't upload a file into the bucket:
./s3 put clibucket/Vol1/test filename=~/test doesn't work.
I enable debugging in libs3 by uncommenting: // curl_easy_setopt_safe(CURLOPT_VERBOSE, 1); and //#define SIGNATURE_DEBUG
here is the trace I got:
got error 422 Unprocessable Entity.
According to Oracle documentation (https://docs.oracle.com/en/cloud/iaas/storage-cloud/cssto/error-code-reference-oracle-storage-cloud-service.html):
422 Unprocessable Entity Cause The value of the ETag header specified in the upload request doesn’t match the MD5 checksum of the HTTP response. Solution This error may be due to a problem in data transmission. Delete the specified object and try again.
I tried to tweek a bunch of headers like region, Etag, content-length, x-amz-content-sha256, md5, etc. Nothing seems to help. Any hint to get libs3 put to work in this configuration would be greatly appreciated.
I think you should submit an issue to Oracle -- libs3 is known to checksum its requests properly and uploads to Amazon S3 has been working for many, many years now without issue. If Oracle is going to claim to support S3, then I think the burden is on them to be compatible. They should analyze the request and see why it's not working and then consider what it is they are doing that is not compatible with Amazon S3 request handling.
Additionally, they should be informed that there is no ETag header in the upload request, so their error message doesn't even make any sense. See this documentation:
http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT.html
ETag is documented thusly:
"Additionally, you can calculate the MD5 while putting an object to Amazon S3 and compare the returned ETag to the calculated MD5 value."
ETag is a header that Amazon responds with, it is not included in the upload request. It is up to the client to use the ETag to verify the upload if it desires. It makes no sense for the Oracle server to report that the upload request even had an ETag header.
By the way, thank you for the detailed error report and for your attempts to diagnose the issue. If you drive this to some solution with Oracle then that would be much appreciated. Thanks again.
Hi bji, thanks for the quick answer. I agry 100%, including regarding ETag. Still, I managed to upload to their cloud, using another client they recommanded on their doc (s3cmd) . So I figured out I could try to make it work somehow on tweaking the client side. But I've done as you recommanded and ask Oracle for help. I'll report any outcome here asap.
@norb62 I tried s3cmd but couldn't configure it using the documentation provided in above link.
I am getting following error
Retrying failed request: /?delimiter=/ ([Errno 111] Connection refused)
Do we need to configure anything on the server side or only curl provided in the document is sufficient?
Some qn:
- Is secret key 2 necessary ? is there any format for the secret key?
- Is below configuration sufficient for s3cfg ? [default] access_key = myservice-bar value updated host_base = acme.storage.oraclecloud.com value updated host_bucket = acme.storage.oraclecloud.com value updated secret_key = MyS3Key1 value updated signature_v2 = False
Please share if you have any document on this, or any suggestion from you would be very helpful. Many Thanks --satyam
