s3-request-presigner: Difficulty in creating a PUT URL with tags

[spencerwilson]

Describe the bug

It's difficult to create a presigned PUT URL that creates an object with tags.

Your environment

SDK version number

@aws-sdk/[email protected]

Is the issue in the browser/Node.js/ReactNative?

Node.js

Details of the browser/Node.js/ReactNative version

v12.22.1

Steps to reproduce

Suppose you have a 3-byte file containing the ASCII text wuf (hexdump: 777566), and you want to create a presigned URL to upload this file. Initially I tried

const putObjectRequest: PutObjectRequest = {
  Bucket: 'uptrust-spencer-test',
  Key: 'my/cool/object.txt',
  ContentMD5: 'fTRbDyK9f+FJrC8Ulj2+4w==',
  Tagging: 'moon=dog',
};

const command = new PutObjectCommand(putObjectRequest);
const signedUrl = await getSignedUrl(s3Client, command, {
  expiresIn: 3600,
});

Then make the following request. Include Content-MD5 since it's a signed header, but do not include x-amz-tagging as it's not a signed header.

✓ % curl -X PUT -H 'Content-MD5: fTRbDyK9f+FJrC8Ulj2+4w==' "$url" --data wuf

Observed behavior

The upload succeeds, but S3 did not attach tags on the object:

✓ % aws s3api get-object-tagging --bucket uptrust-spencer-test --key my/cool/object.txt
{
    "TagSet": []
}

Expected behavior

The upload is successful and the tags are set as specified in the PutObjectCommandInput.

Screenshots

n/a

Additional context

After a lot of experimentation and reading the code, I found a workaround: Making the following adjustment,

  const signedUrl = await getSignedUrl(s3Client, command, {
    expiresIn: 3600,
    // this is new:
    unhoistableHeaders: new Set(['x-amz-tagging']),
  });

, the following request works as intended:

✓ % curl -X PUT -H 'x-amz-tagging: moon=dog' -H 'Content-MD5: fTRbDyK9f+FJrC8Ulj2+4w==' "$url" --data wuf

✓ % aws s3api get-object-tagging --bucket uptrust-spencer-test --key my/cool/object.txt
{
    "TagSet": [
        {
            "Key": "moon",
            "Value": "dog"
        }
    ]
}

This works because

• In the first attempt the x-amz-tagging header is being hoisted out during moveHeadersToQuery and S3 doesn't support an x-amz-tagging query parameter.
• In the second attempt, we're directing SignatureV4#presign to not hoist x-amz-tagging, and instead consider it a signed header. Then (and only then) will a request that includes the header be authorized.

Suggested changes

I think the main issue is that when I saw the x-amz-tagging query parameter in the presigned URL I took that as an indication that S3 supports that as a method of setting tags during PutObject. Problem is, S3 doesn't actually support that.

A better experience, I think, would be to not hoist x-amz-tagging by default. This would make x-amz-tagging a signed header by default, conveying more directly that people need to include x-amz-tagging in their requests. Note also the similarity to server-side encryption: presumably S3 intentionally does not support a query parameter x-amz-tagging as a substitute for the request header x-amz-tagging. The same may be said for x-amz-server-side-encryption, in which case s3-request-presigner understands this and makes an exception here to ensure that the header's signed by default. Perhaps that filter condition could be expanded to include x-amz-tagging?

[yenfryherrerafeliz]

Hi @spencerwilson thank you for opening this issue and also for the feedback provided. I will mark this to be reviewed so we can address this further.

Thanks!

[samthompsonkennedy]

const signedUrl = await getSignedUrl(s3Client, command, { expiresIn: 3600, // this is new: unhoistableHeaders: new Set(['x-amz-tagging']), });

expiresIn: 3600

This just saved me from the hours of troubleshooting.

In the getObjectCommand and putObjectCommand they use different expiry terminology. If you have an expiry date set in putObjectCommand it fails with:

The request signature we calculated does not match the signature you provided

The expiry needs to be set in the getSignedUrl call, NOT the command params.

I know this is slightly off topic.

[Maxwell2022]

What is the difference between

const url = getSignedUrl(client, putCommand, { ...options });

and

  const presigner = new S3RequestPresigner({
    region: '...',
    sha256: Hash.bind(null, 'sha256'),
    credentials: fromNodeProviderChain(),
    uriEscapePath: false,
  });

  const url = await presigner.presign(new HttpRequest({ ...url, method: 'PUT' }),
    { ...options },
  );

I found the documentation is not really explaining that. Also there are no documented example how to limit mimeType / contentLength with presigner.presign() so I assume this is not possible.

Why not exposing a single function that can do all these things?

[RanVaknin]

Closing as duplicate. tracking cross SDK issue here https://github.com/aws/aws-sdk/issues/782