aws-sdk-java icon indicating copy to clipboard operation
aws-sdk-java copied to clipboard

TransferManager.upload doesn't retain `bucketKeyEnabled` setting if using multipart upload

Open ZhaoMJ opened this issue 10 months ago • 0 comments

Upcoming End-of-Support

  • [X] I acknowledge the upcoming end-of-support for AWS SDK for Java v1 was announced, and migration to AWS SDK for Java v2 is recommended.

Describe the bug

If multipart upload is used, files uploaded by the TransferManager don't have the correct object-level bucket key setting as specified in the PutObjectRequest. It turns out that the bucketKeyEnabled setting is not copied when configuring the InitiateMultipartUploadRequest: https://github.com/aws/aws-sdk-java/blob/34b4aa0668c1973252ef764d4768cf3ebfecc15c/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/transfer/internal/UploadCallable.java#L422-L430

Expected Behavior

Objects uploaded by the TransferManager always have the correct object-level bucket key setting as specified in the PutObjectRequest.

Current Behavior

Objects uploaded by the TransferManager don't have the correct object-level bucket key setting as specified in the PutObjectRequest when multipart upload is used.

Reproduction Steps

  1. have a bucket with bucket-level bucket key disabled
  2. Create a TransferManager with a MultipartUploadThreshold of 8MB.
  3. Upload a file of 12MB using
            PutObjectRequest putObjectRequest = new PutObjectRequest(bucket, key, src)
                    .withSSEAwsKeyManagementParams(sseAwsKeyManagementParams)
                    .withBucketKeyEnabled(true);
  1. Verify that S3 bucket key is used for this file.
  2. Create another TransferManager with a MultipartUploadThreshold of 16MB.
  3. Upload the same file with the same PutObjectRequest using the new TransferManager.
  4. Verify that S3 bucket key is not enabled for this file.

Possible Solution

Copy bucketKeyEnabled from PutObjectRequest to InitiateMultipartUploadRequest, i.e. Add the following to UploadCallable.initiateMultipartUpload:

req.withBucketKeyEnabled(origReq.getBucketKeyEnabled())

Additional Information/Context

No response

AWS Java SDK version used

1.12.466

JDK version used

openjdk version "17.0.9" 2023-10-17 LTS OpenJDK Runtime Environment Zulu17.46+19-CA (build 17.0.9+8-LTS) OpenJDK 64-Bit Server VM Zulu17.46+19-CA (build 17.0.9+8-LTS, mixed mode, sharing)

Operating System and version

macOS 13.6.3

ZhaoMJ avatar Apr 23 '24 00:04 ZhaoMJ