scrapy
scrapy copied to clipboard
GCS feed export could be improved to work with a less permissive role
GCS Feed export could apparently use client.bucket instead of client.get_bucket.
This change would allow the use of "Storage Object Admin" IAM role instead of "Storage Admin" Role.
def _store_in_thread(self, file):
file.seek(0)
from google.cloud.storage import Client
client = Client(project=self.project_id)
- bucket = client.get_bucket(self.bucket_name)
+ bucket = client.bucket(self.bucket_name)
blob = bucket.blob(self.blob_name)
blob.upload_from_file(file, predefined_acl=self.acl)
[link to source]
I was concerned about a scenario where such a chance would break stuff for someone with Storage Admin role but no Storage Object Admin role, but looking at https://cloud.google.com/storage/docs/access-control/iam-roles it does look like the Storage Object Admin permissions are a subset of those of Storage Admin.