skypilot icon indicating copy to clipboard operation
skypilot copied to clipboard

Fail to access public bucket with sky storage

Open Michaelvll opened this issue 3 years ago • 0 comments

Although aws s3 ls s3://imagenet-bucket can list the files in the bucket (not owned by me), when mounting the bucket with the following YAML, raises the AccessDenied Error.

resources:
  cloud: AWS

file_mounts:
  /imagenet:
    source: s3://imagenet-bucket
E 07-15 10:00:58 storage.py:556] Sky could not get StoreType.S3 store with name imagenet-bucket.
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetPublicAccessBlock operation: Access Denied

The above exception was the direct cause of the following exception:

sky.exceptions.StorageBucketGetError: Failed to connect to an existing bucket 'imagenet-bucket'.
Please check if:
  1) the bucket name is taken and/or 
  2) the bucket permissions are not setup correctly. Consider using aws s3 ls imagenet-bucket to debug.

Similar problem also observed by @concretevitamin , at https://github.com/sky-proj/sky/pull/974#pullrequestreview-1040516721

Michaelvll avatar Jul 15 '22 17:07 Michaelvll