aws-sdk-java icon indicating copy to clipboard operation
aws-sdk-java copied to clipboard

Use the PutObjectRequest request and call the withAccessControlList method to add the AccessControlList. As a result, the objects in the bucket on the target end have not changed.

Open Pioneer-Even opened this issue 3 years ago • 4 comments

Describe the bug

I need to migrate the objects in one bucket to another bucket, use the PutObjectRequest request, and call the withAccessControlList method to add the AccessControlList. As a result, the data in the destination bucket is the same as the source, but the permissions are not the same but the default Full- Control

Expected Behavior

The expected result is that the two-end object is consistent, including permissions。

Additional Notes: I don't use any bucket policy, I just hope that the operation of uploading objects (including permissions) can be solved through PutObjectRequest, instead of uploading objects through PutObjectRequest first, and then updating the object ACL through SetObjectAclRequest, because such a time cost is related to the data in the bucket. The scale is proportional, and the cost is too high. Is there an efficient update method?

Current Behavior

The current result is that the two-end objects are consistent, but the permissions are inconsistent. For example, the source-end objects are all READ and READ_ACP, but the target-end objects are all Full-Control,but no error occurs

Reproduction Steps

Below is the code sample for migration between two different accounts, but I also tried to migrate between different buckets in the same account, the result is still the same

`public class PutObjectWithAcl { private static final AmazonS3 SOURCE = new S3Client("ak1", "sk1", "http://xxxx").buildS3Client();

private static final AmazonS3 TARGET = new S3Client("ak2", "sk2", "http://xxxx").buildS3Client();

private static final String SOURCE_BUCKET = "test1";

private static final String TARGET_BUCKET = "test2";

private static final Owner SOURCE_OWNER = SOURCE.getS3AccountOwner();

private static final Owner TARGET_OWNER = TARGET.getS3AccountOwner();

public static void main(String[] args) {
    String nextMarker = null;
    ObjectListing objectListing;
    do {
        ListObjectsRequest listObjectsRequest = new ListObjectsRequest();
        listObjectsRequest.withBucketName(SOURCE_BUCKET).withMarker(nextMarker);
        objectListing = SOURCE.listObjects(listObjectsRequest);
        for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) {
            TARGET.putObject(buildPutObjectRequest(TARGET_BUCKET, objectSummary.getKey(),
                    SOURCE.getObject(SOURCE_BUCKET, objectSummary.getKey()),
                    buildAcl(SOURCE.getObjectAcl(SOURCE_BUCKET, objectSummary.getKey()))));
        }
    } while (objectListing.isTruncated());
}

private static AccessControlList buildAcl(AccessControlList controlList) {
    controlList.withOwner(TARGET_OWNER);
    for (Grant grant : controlList.getGrantsAsList()) {
        changeGrantOwner(grant.getGrantee());
    }
    return controlList;
}

private static void changeGrantOwner(Grantee grantee) {
    if (grantee instanceof CanonicalGrantee && grantee.getIdentifier().equals(SOURCE_OWNER.getId())) {
        grantee.setIdentifier(TARGET_OWNER.getId());
        ((CanonicalGrantee) grantee).setDisplayName(TARGET_OWNER.getDisplayName());
    }
}

private static PutObjectRequest buildPutObjectRequest(String bucket, String key, S3Object s3Object, AccessControlList sourceAcl) {
    PutObjectRequest putObjectRequest = new PutObjectRequest(bucket, key, s3Object.getObjectContent(), s3Object.getObjectMetadata());
    putObjectRequest.withAccessControlList(sourceAcl);
    return putObjectRequest;
}

}`

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

1.12.183

JDK version used

1.8

Operating System and version

windows 10 with IDEA

Pioneer-Even avatar May 09 '22 14:05 Pioneer-Even

@Pioneer-Even thank you for reaching out. I'm a little confused though, are you saying that there's an issue in the source getObjectAcl or in the target putObject? Or both?

debora-ito avatar May 11 '22 01:05 debora-ito

@debora-ito this issue is about putObject

Pioneer-Even avatar May 15 '22 04:05 Pioneer-Even

@debora-ito When I use putObject to upload an object and assign an AccessControlList to the object, the expected result is that the object is uploaded successfully and the AccessControlList is assigned successfully, but the actual result is indeed that the object is uploaded successfully. The AccessControlList of the object does not assign the value we specified, but Full-Control

Pioneer-Even avatar May 15 '22 04:05 Pioneer-Even

@debora-ito hello,Does the above question imply a bug? Can you give me some responses?

Pioneer-Even avatar May 30 '22 13:05 Pioneer-Even

@Pioneer-Even apologies for loosing track of this. Do you still see the issue?

If so, can you enable and provide the verbose wirelogs of the listObject and subsequent PutObjects? I don't believe this is a bug in the SDK, but the wirelogs will show exactly that is the ACL being sent in the putObject request.

Instructions on how to enable verbose wirelogging here - https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/java-dg-logging.html#sdk-net-logging-verbose Please make sure to redact any sensitive data (like access keys).

debora-ito avatar Mar 25 '23 02:03 debora-ito

It looks like this issue has not been active for more than five days. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please add a comment to prevent automatic closure, or if the issue is already closed please feel free to reopen it.

github-actions[bot] avatar Mar 30 '23 03:03 github-actions[bot]