scrapy icon indicating copy to clipboard operation
scrapy copied to clipboard

GCS feed export could be improved to work with a less permissive role

Open chanux opened this issue 3 years ago • 1 comments

GCS Feed export could apparently use client.bucket instead of client.get_bucket.

This change would allow the use of "Storage Object Admin" IAM role instead of "Storage Admin" Role.

def _store_in_thread(self, file):
    file.seek(0)
    from google.cloud.storage import Client
    client = Client(project=self.project_id)
-    bucket = client.get_bucket(self.bucket_name)
+    bucket = client.bucket(self.bucket_name)
    blob = bucket.blob(self.blob_name)
    blob.upload_from_file(file, predefined_acl=self.acl)

[link to source]

chanux avatar Apr 13 '22 04:04 chanux

I was concerned about a scenario where such a chance would break stuff for someone with Storage Admin role but no Storage Object Admin role, but looking at https://cloud.google.com/storage/docs/access-control/iam-roles it does look like the Storage Object Admin permissions are a subset of those of Storage Admin.

Gallaecio avatar Apr 13 '22 17:04 Gallaecio