Custom Headers in Presigned URLs Construction

[kasured]

trafficstars

Python: 2.7.5 Boto: 2.48.0 Boto3: 1.4.7 Botocore: 1.7.19

Query String Request Authentication Alternative specifies the format of the StringToSign that might be followed to create a signature of the pre-signed url. Namely,

Signature = URL-Encode( Base64( HMAC-SHA1( YourSecretAccessKeyID, UTF-8-Encoding-Of( StringToSign ) ) ) );

StringToSign = HTTP-VERB + "\n" +
    Content-MD5 + "\n" +
    Content-Type + "\n" +
    Expires + "\n" +
    CanonicalizedAmzHeaders +
    CanonicalizedResource;    

The use case is to be able to provide a custom header like Content-MD5 to participate in signature construction.

This is perfectly fine with the boto library

import boto 

## [s3]
## use-sigv4 = True
boto.config.load_from_path('.boto')

conn = boto.connect_s3(profile_name = "_profile_name_", host = "s3.amazonaws.com")

presigned_url = conn.generate_url(
	expires_in=3600,
	method='GET',
	bucket='_bucket_name_',
	key='_key_',
	headers={'Content-MD5': 'c3993a7637ae5d90e70106050ef6926f'}
)

Seems like this same feature cannot be achieved with boto3.

Can this be confirmed and what is the reason behind degrading this feature from the new version of the library? May it be expected to return in the future versions?

[kyleknap]

Would not be opposed to adding support for custom headers. Could you elaborate on why you need to set the Content-MD5 for a GET request? Typically you would not need to set that a header for GET's and this can be seen in the documentation you linked where the StringToSign in the example had no Content-MD5 included (i.e. it was just an empty string).

[kasured]

I would rather we do keep separate flavors of the issue described here. I specifically stressed out the possibility to have the signature be customized in terms of adding additional components that participate in the process.

Content-MD5 being described as "meaningless" in the GET requests does not disallow it to be present. After all, empty value still gives you a hash albeit deterministic.

The use case described just brings up some additional custom contract to the process. If say Alice and Bob decide to include the custom header in the process of signing url requests, than Charlie being not a part of that contract will have an additional step to be able to get a pre-signed url working for him.

There are lots of scenarios that might be exampled here, without stressing out a specific header.

In the meantime, other SDKs like AWS Java SDK and a "hand-coded" boto library does support that feature without violating any strict contract from the AWS Signing Interface.

[fifthpostulate]

I have seen a couple requests for this, and would find it useful myself. Any chance of it happening?

[ATGiant]

Ditto. This would be a great feature.

[ayqazi]

Presigned URLs absolutely require custom metadata to be attached, or else how can one trigger a job off an S3 upload, and keep track of it without attaching a UUID metadatum to it?

[dmyur]

I wish I could add custom headers to my presigned URLs with boto3 😭

[pnminh]

have we got any good news yet? I'm desperately in need with some extra custom headers I want to add.

[Dizzzmas]

Would really like to see those added. Could allow passing custom metadata to the uploaded objects https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html#object-metadata

[github-actions[bot]]

Greetings! It looks like this issue hasn’t been active in longer than one year. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.

[yanivpaz]

The issue still exist ....

[yanivpaz]

see also https://github.com/boto/boto3/discussions/3342