cloudpathlib icon indicating copy to clipboard operation
cloudpathlib copied to clipboard

failing to write to S3 bucket without read permission

Open pconstr opened this issue 5 months ago • 5 comments

I've got a use case where I'd like to use cloudpathlib to write to a bucket where the user has permissions for put_object but not for get_object.

Currently (0.19) I get an exception when opening as it appears cloudpathlib attempts to refresh the cache:

   with (target / f"{k}.csv").open("w") as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/cloudpathlib/cloudpath.py", line 663, in open
    self._refresh_cache(force_overwrite_from_cloud=force_overwrite_from_cloud)
  File "/usr/local/lib/python3.11/dist-packages/cloudpathlib/cloudpath.py", line 1220, in _refresh_cache
    stats = self.stat()
            ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/cloudpathlib/s3/s3path.py", line 61, in stat
    meta = self.client._get_metadata(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/cloudpathlib/s3/s3client.py", line 139, in _get_metadata
    data = self.s3.ObjectSummary(cloud_path.bucket, cloud_path.key).get(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/boto3/resources/factory.py", line 581, in do_action
    response = action(self, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/boto3/resources/action.py", line 88, in __call__
    response = getattr(parent.meta.client, operation_name)(*args, **params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/botocore/client.py", line 569, in _api_call
    return self._make_api_call(operation_name, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/botocore/client.py", line 1023, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetObject operation: User: arn:aws:iam::************:user/************ is not authorized to perform: s3:GetObject on resource: "arn:aws:s3:::********.csv" because no identity-based policy allows the s3:GetObject action

Is there a way to work around this? If there isn't but it's something that could be changed, I'd be happy to work on a patch given some direction as to what would be acceptable.

pconstr avatar Sep 18 '24 21:09 pconstr