getSignedUrl Doesn't work with ContentDisposition param of PutObjectCommand

[harissarwar]

Checkboxes for prior research

• [X] I've gone through Developer Guide and API reference
• [X] I've checked AWS Forums and StackOverflow.
• [X] I've searched for previous similar issues and didn't find any solution.

Describe the bug

I am trying to generate pre signed url for PUT in S3 in an effort to migrate from V2 to V3. I am able to upload via pre signed url if PutObjectCommand has following parameters. const params = { Bucket: "abc.xyz.com", Key: "image.jpg", ContentType: 'image/jpeg', StorageClass: 'REDUCED_REDUNDANCY', ACL: 'private', }; Presigned url gives signature mismatch error if any of the commented out parameter is added in PutObjectCommand const params = { Bucket: "abc.xyz.com", Key: "image.jpg", StorageClass: 'REDUCED_REDUNDANCY', ACL: 'private', ContentType: 'image/jpeg', // ContentDisposition: 'attachment', // ServerSideEncryption: 'AES256', }; For example presigned url generated by following parameter will generate signature mismatch error. const params = { Bucket: "abc.xyz.com", Key: "image.jpg", ContentType: 'image/jpeg', StorageClass: 'REDUCED_REDUNDANCY', ACL: 'private', ContentDisposition: 'attachment', };

SDK version number

@aws-sdk/[email protected],@aws-sdk/[email protected]

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

v16.20.2

Reproduction Steps

const { getSignedUrl } = require("@aws-sdk/s3-request-presigner");
const { S3Client, PutObjectCommand } = require("@aws-sdk/client-s3");
const { createReadStream, statSync } = require("fs");

const region = 'us-east-1';
const filePath = 'image.jpeg';

const s3Client = new S3Client({ region });

const getPresSignedUrl = async () => {
  const params = {
    Bucket: "abc.xyz.com",
    Key: "image.jpg", 
    ContentType: 'image/jpeg',
    StorageClass: 'REDUCED_REDUNDANCY',
    ACL: 'private',
    ContentDisposition: 'attachment',
    // ServerSideEncryption: 'AES256',
  };

  const command = new PutObjectCommand(params);
  const url = await getSignedUrl(s3Client, command, { expiresIn: 3600 });
  return url;
}

const upload = async (url) => {
  const { default: fetch } = await import("node-fetch");
  const payload = createReadStream(filePath);

  const response = await fetch(url, {
    method: "PUT",
    body: payload, 
    headers: {
      "Content-Length": statSync(filePath).size
    }
  })
  
  return { 
    status: response.status,
    statusText: response.statusText,
    data: await response.text()
  }
}

const test = async () => {
  const url = await getPresSignedUrl();
  const result = await upload(url);
  return {
    url,
    result 
  }
}

test().then(({ url, result }) => {
  console.log('curl -X PUT -T image.jpeg "${url}"');
  console.log('upload result', result);
});


### Observed Behavior

<?xml version="1.0" encoding="UTF-8"?>\n' +
    '<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>XXXXXXXXXX</AWSAccessKeyId>......

### Expected Behavior

We should be able to upload file with generated pre signed url.

### Possible Solution

_No response_

### Additional Information/Context

I am seeing this issue if any of the following parameter is added to PutObjectCommand.
- ContentDisposition: 'attachment'
- ServerSideEncryption: 'AES256'

[aBurmeseDev]

Hi @harissarwar - thanks for reaching out.

When you're generating Presigned URL, SDK is only responsible for generating the URL and not for sending the request to S3. In this scenario, you'll need to pass corresponding headers through your request such as x-amz-server-side-encryption. Reference here: https://docs.aws.amazon.com/AmazonS3/latest/userguide/specifying-kms-encryption.html

It's also noted in our API reference docs:

If your request contains server-side encryption(SSE*) configurations, because of S3 limitation, you need to send corresponding headers along with the presigned url.

Hope it helps but let me know if issue persists. Best. John

[harissarwar]

Hi aBurmeseDev, thanks for the update. I also see this issue when I pass ContentDisposition. I don't see anything in the documentation related to this. The sample code I shared will generate an error and its not using ServerSideEncryption parameter.

[aBurmeseDev]

Can you share you updated code? Also add this middleware stack to your client and share the outputs? Middleware would give us raw request of the client call.

client.middlewareStack.add(
  (next, context) => async (args) => {
    console.log("AWS SDK context", context.clientName, context.commandName);
    console.log("AWS SDK request input", args.input);
    const result = await next(args);
    console.log("AWS SDK request output:", result.output);
    return result;
  },
  {
    name: "MyMiddleware",
    step: "build",
    override: true,
  }
);

[harissarwar]

Actually I am not making any client call using sdk. I am just generating a pre signed url, which I don't think generates a call to AWS.

Presigned url is then used by fetch to upload the file which doesn't uses sdk.

[harissarwar]

I added the middleware and here are the logs. In AWS SDK V2 there was not server call. It seem to be making call to server in V3. I have attached the sample code.

sample.zip

Please note, I have changed the bucket name and path in the logs below.

AWS SDK context S3Client PutObjectCommand
AWS SDK request input {
  Bucket: 'xyz.abc.com',
  Key: 'image.jpeg',
  ContentType: 'image/jpeg',
  StorageClass: 'REDUCED_REDUNDANCY',
  ACL: 'private',
  ContentDisposition: 'attachment'
}
AWS SDK request output: {
  '$metadata': { httpStatusCode: 200, attempts: 1, totalRetryDelay: 0 },
  presigned: {
    method: 'PUT',
    hostname: 's3.us-east-1.amazonaws.com',
    port: undefined,
    body: undefined,
    protocol: 'https:',
    path: '/xyz.abc.com/image.jpeg',
    username: undefined,
    password: undefined,
    fragment: undefined,
    headers: {
      'content-type': 'image/jpeg',
      'content-disposition': 'attachment',
      host: 's3.us-east-1.amazonaws.com',
      'user-agent': 'aws-sdk-js/3.574.0 ua/2.0 os/darwin#23.2.0 lang/js md/nodejs#16.20.2 api/s3#3.574.0'
    },
    query: {
      'x-id': 'PutObject',
      'x-amz-acl': 'private',
      'x-amz-storage-class': 'REDUCED_REDUNDANCY',
      'X-Amz-Content-Sha256': 'UNSIGNED-PAYLOAD',
      'X-Amz-Algorithm': 'AWS4-HMAC-SHA256',
      'X-Amz-Credential': 'AKIAQ4GMKQ5OSNBQDJBS/20240511/us-east-1/s3/aws4_request',
      'X-Amz-Date': '20240511T023042Z',
      'X-Amz-Expires': '3600',
      'X-Amz-SignedHeaders': 'content-disposition;host',
      'X-Amz-Signature': 'c8e6182054ff0d7e0b219de5e7a9597b87821d38227e7e4f9027d949eaa61f55'
    }
  }
}
curl -X PUT -T image.jpeg "https://s3.us-east-1.amazonaws.com/xyz.abc.com/image.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAQ4GMKQ5OSNBQDJBS%2F20240511%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240511T023042Z&X-Amz-Expires=3600&X-Amz-Signature=c8e6182054ff0d7e0b219de5e7a9597b87821d38227e7e4f9027d949eaa61f55&X-Amz-SignedHeaders=content-disposition%3Bhost&x-amz-acl=private&x-amz-storage-class=REDUCED_REDUNDANCY&x-id=PutObject"
upload result {
  status: 403,
  statusText: 'Forbidden',
  data: '<?xml version="1.0" encoding="UTF-8"?>\n' +
    '<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>AKIAQ4GMKQ5OSNBQDJBS</AWSAccessKeyId><StringToSign>AWS4-HMAC-SHA256\n' +
    '20240511T023042Z\n' +
    '20240511/us-east-1/s3/aws4_request\n' +
    'e7c4e711e8a500517a9fcabfb85531fcf4d783afd664955b4075161bd3dd8fa0</StringToSign><SignatureProvided>c8e6182054ff0d7e0b219de5e7a9597b87821d38227e7e4f9027d949eaa61f55</SignatureProvided><StringToSignBytes>41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 32 34 30 35 31 31 54 30 32 33 30 34 32 5a 0a 32 30 32 34 30 35 31 31 2f 75 73 2d 65 61 73 74 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 65 37 63 34 65 37 31 31 65 38 61 35 30 30 35 31 37 61 39 66 63 61 62 66 62 38 35 35 33 31 66 63 66 34 64 37 38 33 61 66 64 36 36 34 39 35 35 62 34 30 37 35 31 36 31 62 64 33 64 64 38 66 61 30</StringToSignBytes><CanonicalRequest>PUT\n' +
    '/xyz.abc.com/image.jpeg\n' +
    'X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&amp;X-Amz-Credential=AKIAQ4GMKQ5OSNBQDJBS%2F20240511%2Fus-east-1%2Fs3%2Faws4_request&amp;X-Amz-Date=20240511T023042Z&amp;X-Amz-Expires=3600&amp;X-Amz-SignedHeaders=content-disposition%3Bhost&amp;x-amz-acl=private&amp;x-amz-storage-class=REDUCED_REDUNDANCY&amp;x-id=PutObject\n' +
    'content-disposition:\n' +
    'host:s3.us-east-1.amazonaws.com\n' +
    '\n' +
    'content-disposition;host\n' +
    'UNSIGNED-PAYLOAD</CanonicalRequest><CanonicalRequestBytes>50 55 54 0a 2f 7a 69 67 72 6f 6e 2e 67 6f 61 62 6f 64 65 2e 63 6f 6d 2f 6d 65 64 69 61 2f 38 36 65 31 38 36 38 31 37 32 35 34 34 62 32 62 61 34 37 61 36 33 32 37 36 66 33 30 36 34 37 65 2f 62 30 63 35 63 61 33 35 64 66 35 33 2f 32 30 32 34 2d 30 35 2d 30 31 2f 62 30 63 35 63 61 33 35 64 66 35 33 5f 32 30 32 34 2d 30 35 2d 30 31 5f 31 33 35 34 31 32 5f 30 2e 6a 70 67 0a 58 2d 41 6d 7a 2d 41 6c 67 6f 72 69 74 68 6d 3d 41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 26 58 2d 41 6d 7a 2d 43 6f 6e 74 65 6e 74 2d 53 68 61 32 35 36 3d 55 4e 53 49 47 4e 45 44 2d 50 41 59 4c 4f 41 44 26 58 2d 41 6d 7a 2d 43 72 65 64 65 6e 74 69 61 6c 3d 41 4b 49 41 51 34 47 4d 4b 51 35 4f 53 4e 42 51 44 4a 42 53 25 32 46 32 30 32 34 30 35 31 31 25 32 46 75 73 2d 65 61 73 74 2d 31 25 32 46 73 33 25 32 46 61 77 73 34 5f 72 65 71 75 65 73 74 26 58 2d 41 6d 7a 2d 44 61 74 65 3d 32 30 32 34 30 35 31 31 54 30 32 33 30 34 32 5a 26 58 2d 41 6d 7a 2d 45 78 70 69 72 65 73 3d 33 36 30 30 26 58 2d 41 6d 7a 2d 53 69 67 6e 65 64 48 65 61 64 65 72 73 3d 63 6f 6e 74 65 6e 74 2d 64 69 73 70 6f 73 69 74 69 6f 6e 25 33 42 68 6f 73 74 26 78 2d 61 6d 7a 2d 61 63 6c 3d 70 72 69 76 61 74 65 26 78 2d 61 6d 7a 2d 73 74 6f 72 61 67 65 2d 63 6c 61 73 73 3d 52 45 44 55 43 45 44 5f 52 45 44 55 4e 44 41 4e 43 59 26 78 2d 69 64 3d 50 75 74 4f 62 6a 65 63 74 0a 63 6f 6e 74 65 6e 74 2d 64 69 73 70 6f 73 69 74 69 6f 6e 3a 0a 68 6f 73 74 3a 73 33 2e 75 73 2d 65 61 73 74 2d 31 2e 61 6d 61 7a 6f 6e 61 77 73 2e 63 6f 6d 0a 0a 63 6f 6e 74 65 6e 74 2d 64 69 73 70 6f 73 69 74 69 6f 6e 3b 68 6f 73 74 0a 55 4e 53 49 47 4e 45 44 2d 50 41 59 4c 4f 41 44</CanonicalRequestBytes><RequestId>XHQCKEW8AP4NF75K</RequestId><HostId>unbZAxEGy+et/sDaLlOSe7brLKCejee0muZalKsnWK6cwbbpe59NXYGrT1P8x3hxGRObDpLXiLE=</HostId></Error>'
}

[RanVaknin]

Hi @harissarwar ,

If you look at your presigned URL:

https://s3.us-east-1.amazonaws.com/xyz.abc.com/image.jpeg
?X-Amz-Algorithm=AWS4-HMAC-SHA256
&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD
&X-Amz-Credential=REDACTED/20240511/us-east-1/s3/aws4_request
&X-Amz-Date=20240511T023042Z
&X-Amz-Expires=3600
&X-Amz-Signature=REDACTED
&X-Amz-SignedHeaders=content-disposition;host
&x-amz-acl=private
&x-amz-storage-class=REDUCED_REDUNDANCY
&x-id=PutObject

Notice that under signedHeaders you have content-disposition, and host. This means that the signature was calculated with the intent of those headers being included in the request (host is usually included by default)

In your code:

  const response = await fetch(url, {
    method: "PUT",
    body: payload, 
    headers: {
      "Content-Length": statSync(filePath).size
    }
  })

You are only sending content length, which was also not presigned.

I would try to parse your generated presigned URL, and programmatically copy the signedHeaders provided in the URL to the actual fetch request.

As a side note, I would also use something other than node-fetch since it does not provide an easy way to inspect the outgoing raw requests. Something like Axios will let you do that, and compare between the raw request sent and the generated signed URL.

Thanks, Ran~

[harissarwar]

Hi @RanVaknin, are you saying that I should pass the content-disposition header when using pre signed url to upload something?

I am passing this presigned url to a camera so it can upload logs to s3. I don't want to make any change in the firmware how camera uses pre signed url. On the server side I have migrated from v2 to v3, this should not affect how client uses the presigned url. I was generating pre signed url in V2 in similar way and did not see this issue.

[RanVaknin]

Hi @harissarwar ,

Hi @RanVaknin, are you saying that I should pass the content-disposition header when using pre signed url to upload something?

Yes.

I am passing this presigned url to a camera so it can upload logs to s3. I don't want to make any change in the firmware how camera uses pre signed url. On the server side I have migrated from v2 to v3, this should not affect how client uses the presigned url. I was generating pre signed url in V2 in similar way and did not see this issue.

In v3 to conform with s3 specific requirements we are explicitly presigning certain headers.

You can try to force the presigner to not sign this header:

  return getSignedUrl(client, command, { 
    expiresIn: 3600,
    unsignableHeaders: new Set(['content-disposition' ])
   });

Thanks, Ran~

[harissarwar]

Thanks @RanVaknin, It's working now after using unsignableHeaders.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.