aws-sdk-java icon indicating copy to clipboard operation
aws-sdk-java copied to clipboard
verified publisher icon aws

deleteObject ignores isRequesterPays

Open jfromme opened this issue 3 years ago • 3 comments

Describe the bug

The deleteObject methods in AmazonS3Client do not check or use the request's isRequesterPays value.

Expected Behavior

When a delete object request has isRequesterPays set to true and the target bucket has requester pays enabled then the request should succeed assuming the user has the required permsisions.

Current Behavior

Instead the request fails with an AccessDenied error.

Reproduction Steps

Attempt to delete an object from a requester pays enabled bucket.

Possible Solution

deleteObject should call the populateRequesterPaysHeader as other methods in the client do.

Additional Information/Context

No response

AWS Java SDK version used

1.11.931

JDK version used

8u212

Operating System and version

Alpine Linux 3.9

jfromme avatar Oct 06 '22 19:10 jfromme

Are you sure the requester has the necessary permissions to access the object? AccessDenied indicates it has not. Are you providing the requesterPays parameter in DeleteObjectRequest? Can you share a repro code?

debora-ito avatar Oct 08 '22 00:10 debora-ito

I believe the permissions are okay. Switching from using deleteObject() to deleteObjects() worked without any modifications to the user or bucket permissions.

Here is a test case which uses MockServer to set expectations on the request being sent. testDeleteObject() fails with a 404 because the expected request header is absent. In contrast, testDeleteObjects() passes. https://github.com/jfromme/delete-object-test

Here is the MockServer log from testDeleteObject():

2022-10-10 16:27:40 5.14.0 INFO 52920 started on port: 52920 
2022-10-10 16:27:41 5.14.0 INFO 52920 creating expectation:

  {
    "httpRequest" : {
      "method" : "DELETE",
      "path" : "/test-bucket/file-key",
      "headers" : {
        "x-amz-request-payer" : [ "requester" ]
      }
    },
    "httpResponse" : {
      "statusCode" : 204
    },
    "id" : "3555c98f-28b1-4216-8d62-68f13223e655",
    "priority" : 0,
    "timeToLive" : {
      "unlimited" : true
    },
    "times" : {
      "unlimited" : true
    }
  }

 with id:

  3555c98f-28b1-4216-8d62-68f13223e655
 
2022-10-10 16:27:41 5.14.0 INFO 52920 received request:

  {
    "method" : "DELETE",
    "path" : "/test-bucket/file-key",
    "headers" : {
      "x-amz-content-sha256" : [ "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" ],
      "content-length" : [ "0" ],
      "content-encoding" : [ ".*" ],
      "amz-sdk-retry" : [ "0/0/500" ],
      "amz-sdk-request" : [ "attempt=1;max=4" ],
      "amz-sdk-invocation-id" : [ "ae9d22c7-80eb-f7f7-99e0-0e6df6702135" ],
      "X-Amz-Date" : [ "20221010T202741Z" ],
      "User-Agent" : [ "aws-sdk-java/1.12.319 Mac_OS_X/10.15.7 OpenJDK_64-Bit_Server_VM/25.312-b07 java/1.8.0_312 vendor/Amazon.com_Inc. cfg/retry-mode/legacy" ],
      "Host" : [ "localhost:52920" ],
      "Content-Type" : [ "application/octet-stream" ],
      "Connection" : [ "Keep-Alive" ],
      "Authorization" : [ "AWS4-HMAC-SHA256 Credential=AKIA5SOSPFND4TX4E4Y7/20221010/us-east-1/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;amz-sdk-retry;content-type;host;user-agent;x-amz-content-sha256;x-amz-date, Signature=b8d6c98498f9bc24ff46a4022826363e914d595f234aeecfb2d08ab77b800cdc" ]
    },
    "keepAlive" : true,
    "secure" : false,
    "localAddress" : "127.0.0.1:52920",
    "remoteAddress" : "127.0.0.1"
  }
 
2022-10-10 16:27:41 5.14.0 INFO 52920 request:

  {
    "method" : "DELETE",
    "path" : "/test-bucket/file-key",
    "headers" : {
      "x-amz-content-sha256" : [ "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" ],
      "content-length" : [ "0" ],
      "content-encoding" : [ ".*" ],
      "amz-sdk-retry" : [ "0/0/500" ],
      "amz-sdk-request" : [ "attempt=1;max=4" ],
      "amz-sdk-invocation-id" : [ "ae9d22c7-80eb-f7f7-99e0-0e6df6702135" ],
      "X-Amz-Date" : [ "20221010T202741Z" ],
      "User-Agent" : [ "aws-sdk-java/1.12.319 Mac_OS_X/10.15.7 OpenJDK_64-Bit_Server_VM/25.312-b07 java/1.8.0_312 vendor/Amazon.com_Inc. cfg/retry-mode/legacy" ],
      "Host" : [ "localhost:52920" ],
      "Content-Type" : [ "application/octet-stream" ],
      "Connection" : [ "Keep-Alive" ],
      "Authorization" : [ "AWS4-HMAC-SHA256 Credential=AKIA5SOSPFND4TX4E4Y7/20221010/us-east-1/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;amz-sdk-retry;content-type;host;user-agent;x-amz-content-sha256;x-amz-date, Signature=b8d6c98498f9bc24ff46a4022826363e914d595f234aeecfb2d08ab77b800cdc" ]
    },
    "keepAlive" : true,
    "secure" : false,
    "localAddress" : "127.0.0.1:52920",
    "remoteAddress" : "127.0.0.1"
  }

 didn't match expectation:

  {
    "httpRequest" : {
      "method" : "DELETE",
      "path" : "/test-bucket/file-key",
      "headers" : {
        "x-amz-request-payer" : [ "requester" ]
      }
    },
    "httpResponse" : {
      "statusCode" : 204
    },
    "id" : "3555c98f-28b1-4216-8d62-68f13223e655",
    "priority" : 0,
    "timeToLive" : {
      "unlimited" : true
    },
    "times" : {
      "unlimited" : true
    }
  }

 because:

  method matched
  path matched
  body matched
  headers didn't match
 
2022-10-10 16:27:41 5.14.0 INFO 52920 no expectation for:

  {
    "method" : "DELETE",
    "path" : "/test-bucket/file-key",
    "headers" : {
      "x-amz-content-sha256" : [ "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" ],
      "content-length" : [ "0" ],
      "content-encoding" : [ ".*" ],
      "amz-sdk-retry" : [ "0/0/500" ],
      "amz-sdk-request" : [ "attempt=1;max=4" ],
      "amz-sdk-invocation-id" : [ "ae9d22c7-80eb-f7f7-99e0-0e6df6702135" ],
      "X-Amz-Date" : [ "20221010T202741Z" ],
      "User-Agent" : [ "aws-sdk-java/1.12.319 Mac_OS_X/10.15.7 OpenJDK_64-Bit_Server_VM/25.312-b07 java/1.8.0_312 vendor/Amazon.com_Inc. cfg/retry-mode/legacy" ],
      "Host" : [ "localhost:52920" ],
      "Content-Type" : [ "application/octet-stream" ],
      "Connection" : [ "Keep-Alive" ],
      "Authorization" : [ "AWS4-HMAC-SHA256 Credential=AKIA5SOSPFND4TX4E4Y7/20221010/us-east-1/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;amz-sdk-retry;content-type;host;user-agent;x-amz-content-sha256;x-amz-date, Signature=b8d6c98498f9bc24ff46a4022826363e914d595f234aeecfb2d08ab77b800cdc" ]
    },
    "keepAlive" : true,
    "secure" : false,
    "localAddress" : "127.0.0.1:52920",
    "remoteAddress" : "127.0.0.1"
  }

 returning response:

  {
    "statusCode" : 404,
    "reasonPhrase" : "Not Found"
  }

jfromme avatar Oct 10 '22 20:10 jfromme

@jfromme apologies for the long silence.

Yes, I can reproduce, the x-amz-requester-payer is not being added to the request. The workaround is to use DeleteObjects.

debora-ito avatar Apr 01 '23 02:04 debora-ito

Hi @jfromme

We apologize but this won't get fixed in v1 before going into Maintenance Mode. The workaround is to use DeleteObjects (plural) if you need to enable isRequesterPays.

Reference:

  • Announcing end-of-support for AWS SDK for Java v1.x effective December 31, 2025 - blog post

debora-ito avatar Jun 20 '24 01:06 debora-ito

This issue is now closed.

Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.

github-actions[bot] avatar Jun 20 '24 01:06 github-actions[bot]