GCS Bucket Mover - bucket policy only

[wmuizelaar]

Hi,

I love the GCS Bucket Mover tool - but would it be possible to use it in combination with the bucket policy only feature? It looks like it now really depends on ACLs to exist or at least be accessible, which isn't the case when bucket policy only is activated.

Thanks,

Wietse

[lukwam]

I am having the same issue. Even if I add the --skip_acl option, it still attempts to pull the ACLs here:

https://github.com/GoogleCloudPlatform/professional-services/blob/main/tools/gcs-bucket-mover/gcs_bucket_mover/bucket_details.py#L54

which is calling get_entities on the storage_object.acl object:

https://googleapis.dev/python/storage/latest/_modules/google/cloud/storage/acl.html#ACL.get_entities

which fails with the error:

google.api_core.exceptions.BadRequest: 400 GET https://www.googleapis.com/storage/v1/b/my-gcs-bucket-mover-test-bucket/acl: Cannot get legacy ACL for a bucket that has uniform bucket-level access. Read more at https://cloud.google.com/storage/docs/uniform-bucket-level-access

which is the expected behavior when uniform bucket access is enabled.

https://cloud.google.com/storage/docs/uniform-bucket-level-access#enabled

I have found that if I first change the bucket to use Fine Grained Access, then the script is able to run.

[bsikander]

Is there any update on this? I am having the exact issue as above.

[markueez]

@bsikander @lukwam it seems fixed in the latest commits... however, since I had to fix up py3 compatibility anyway, see my fork @ https://github.com/GoogleCloudPlatform/professional-services/compare/main...markueez:professional-services:py3 -- I just migrated tons of buckets with uniform and obj-level ACL's and everything went OK

[bsikander]

@markueez is this the commit which fixes this and allows us to migrate uniform buckets?

[markueez]

yes, specifically, https://github.com/GoogleCloudPlatform/professional-services/pull/878/commits/354c58637ecf037022f71659bf35d75732300636#diff-8458a8a84d96e3d265dc65e57007c571525af35dcfceb1732d902fd230bd192fR218

On Mon, Oct 10, 2022, 14:20 Behroz Sikander @.***> wrote:

@markueez https://github.com/markueez is this https://github.com/GoogleCloudPlatform/professional-services/pull/878/ the commit which fixes this and allows us to migrate uniform buckets?

— Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/professional-services/issues/327#issuecomment-1273163347, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZTBR7R5KMJJIRDVIKWS323WCP3ZPANCNFSM4I5AYQZQ . You are receiving this because you were mentioned.Message ID: @.*** com>

[bsikander]

cool thanks. Will try it out.

[agold-rh]

@markueez Do you have any code that should be merged, or should this be closed?

[markueez]

@agold-rh I think this can be closed (worked for me + original reporter never came back)