cloudpathlib
cloudpathlib copied to clipboard
failing to write to S3 bucket without read permission
I've got a use case where I'd like to use cloudpathlib to write to a bucket where the user has permissions for put_object but not for get_object.
Currently (0.19) I get an exception when opening as it appears cloudpathlib attempts to refresh the cache:
with (target / f"{k}.csv").open("w") as f:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/cloudpathlib/cloudpath.py", line 663, in open
self._refresh_cache(force_overwrite_from_cloud=force_overwrite_from_cloud)
File "/usr/local/lib/python3.11/dist-packages/cloudpathlib/cloudpath.py", line 1220, in _refresh_cache
stats = self.stat()
^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/cloudpathlib/s3/s3path.py", line 61, in stat
meta = self.client._get_metadata(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/cloudpathlib/s3/s3client.py", line 139, in _get_metadata
data = self.s3.ObjectSummary(cloud_path.bucket, cloud_path.key).get(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/boto3/resources/factory.py", line 581, in do_action
response = action(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/boto3/resources/action.py", line 88, in __call__
response = getattr(parent.meta.client, operation_name)(*args, **params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/botocore/client.py", line 569, in _api_call
return self._make_api_call(operation_name, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/botocore/client.py", line 1023, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetObject operation: User: arn:aws:iam::************:user/************ is not authorized to perform: s3:GetObject on resource: "arn:aws:s3:::********.csv" because no identity-based policy allows the s3:GetObject action
Is there a way to work around this? If there isn't but it's something that could be changed, I'd be happy to work on a patch given some direction as to what would be acceptable.