[BUG] ESOP fails with GCP uniform bucket-level access enabled - predefinedAcl

[pushkar-anand]

Describe the bug ESOP backup fails when using Google Cloud Storage buckets with uniform bucket-level access enabled. The error occurs because ESOP automatically sends predefinedAcl=bucketOwnerFullControl parameter during upload operations, which is incompatible with uniform bucket-level access that disables all ACL functionality.

To Reproduce Steps to reproduce the behavior:

1. Create a GCS bucket with uniform bucket-level access enabled:

gsutil mb gs://test-bucket
gsutil uniformbucketlevelaccess set on gs://test-bucket

2. Configure proper IAM permissions for your service account on the bucket

3. Trigger backup from Icarus

4. Observe the error:

PUT https://storage.googleapis.com/upload/storage/v1/b/test-bucket/o?name=...&predefinedAcl=bucketOwnerFullControl&uploadType=resumable&upload_id=...
{
  "error": {
    "code": 400,
    "message": "Cannot insert legacy ACL for an object when uniform bucket-level access is enabled. Read more at https://cloud.google.com/storage/docs/uniform-bucket-level-access",
    "errors": [
      {
        "message": "Cannot insert legacy ACL for an object when uniform bucket-level access is enabled. Read more at https://cloud.google.com/storage/docs/uniform-bucket-level-access",
        "domain": "global",
        "reason": "invalid"
      }
    ]
  }
}

Expected behavior

ESOP should successfully upload files to GCS buckets with uniform bucket-level access enabled.

System and versions (please complete the following information):

java -jar /usr/share/java/icarus.jar esop --version
instaclustr-esop 3.0.0
Build time: 2024-10-02T00:03:24+0200
Git commit: unknown

java -jar /usr/share/java/icarus.jar icarus --version
instaclustr-icarus 3.0.0
Build time: 2024-10-02T00:03:24+0200
Git commit: unknown

Additional context

• Google Cloud Storage uniform bucket-level access is the recommended security practice and is often enforced by organization policies
• Similar issues have been reported in other libraries (e.g., googleapis/nodejs-storage#1230, thephpleague/flysystem#1357)

[pushkar-anand]

@smiklosovic I have a rough implementation for the fix in #91.

Does it make sense to have a flag --gcp-uniform-bucket-level-access that skips adding predefined ACL policies, or do you have any other ideas on fixing this?

[smiklosovic]

@pushkar-anand fine to introduce a flag like that.