Cannot presign url for s3 delete_objects operation

[NyanHelsing]

So far as I can tell, the endpoint requires a query parameter, delete in the url. See https://docs.aws.amazon.com/AmazonS3/latest/API/multiobjectdeleteapi.html for documentation. generate_presigned_post has no way to provide this query parameter, and so cannot correctly sign a url to be used with this api.

generate_presigned_url does not work either, as it has no way to add what the hash of the body will be when signing the url.

Hopefully I am missing something here.

[NyanHelsing]

think this is the relevant code: https://github.com/boto/botocore/blob/939fbebcae0e0519f040063d61f7dc354cd09dd8/botocore/signers.py#L681-L682

[JordonPhillips]

generate_presigned_post is specifically for the put_objects operation, what you want to use is generate_presigned_url with the POST method. The difference is the way that the signatures are generated, afaik S3 only supports PutObject with the presigned_post way of signing.

[NyanHelsing]

That's the case I found, and what I've ended up using. There is still a problem here however, generate_presigned_url doesn't have a way to pass headers in like generate_presigned_post does.

[NyanHelsing]

We were forced to write a modified version of this function in order to sign delete operations. https://github.com/CenterForOpenScience/waterbutler/pull/351/files#diff-74bbac431e282383494483a4d0af3031R48

[NyanHelsing]

See https://github.com/boto/botocore/pull/1516 for a pr that allows headers to be passed into the url signing process.

[NyanHelsing]

Bump for a response

[swetashre]

@ExProbitasFiducia - There is no parameter for custom headers in boto3's generate_presigned_url. The headers get added based on the parameters that you would normally pass to the client's method.

Please let me know if you have any questions.

[no-response[bot]]

This issue has been automatically closed because there has been no response to our request for more information from the original author. With only the information that is currently in the issue, we don't have enough information to take action. Please reach out if you have or find the answers we need so that we can investigate further.

[NyanHelsing]

@tim-finnigan appreciate the follow up on this! iirc the url generated here does not include any headers that were passed to it, so the signature generated by boto cannot be validated by s3, because when s3 checks the signature, it does take into account headers.

this isn't about enabling custom headers i think it was to make the functionality work at all.

[NyanHelsing]

https://github.com/CenterForOpenScience/waterbutler/pull/351/files#diff-e343fe1cb1d426f8d1f366e29e565554bb2d55cd77055b69e92e94b776d1216aR48-R114 for a use case/ link to the line numbers with the monkeypatch i'd written

in file waterbutler/providers/s3/provider.py ln 48 -114

[NyanHelsing]

@felliott @mfraezz @johnetordoff might be interested in any progress here.