403 Forbidden with user without list all buckets right

[nelgo]

Good Morning,

i have an issue working with this library while using an user without the right to list all existing bucket. If I got it right, every time I try to retrieve a Bucket, the library first tries to list all the existing buckets and then picks the one requested by the user, causing a 403 Forbidden error when using a "limited access" user.

Looking at how the AWS Java client deal with this situation by providing a direct method to retrieve an object by providing a bucket name and file path, i think it could be enough to add a similar method to get around the problem. For example, looking at getObject, i think adding a new

def getObject(bucketName: String, key: String)

could go.

Unfortunately I am not an S3 expert to evaluate all the possible implication, i just hope this to be useful.

What do you think?

Thanks for your work

[seratch]

Maybe that would be useful for others.

[krotton]

:+1:

There's also S3.bucket(name: String): Option[Bucket] which needlessly uses listBuckets - that means an additional roundtrip and requires permissions to list buckets even though access permissions to a single bucket would suffice.

[crakjie]

👍 +1

[maartenh]

Ran into this as well, and worked around it by just creating a Bucket instance directly:

val bucket: Bucket = Bucket("some-bucket-name")

[adisesha]

There's also S3.bucket(name: String): Option[Bucket] which needlessly uses listBuckets - that means an additional roundtrip and requires permissions to list buckets even though access permissions to a single bucket would suffice.

This needless operation can cause extra AWS bills, depending on the frequency. In our case 1000+ dollars. Calls to list buckets can result in cross region data transfers and requests. Checkout for ListAllMyBuckets in usage reports or ListBuckets in Cloudtrail logs.

[Bertrand31]

Wouldn't this method be made a lot more efficient by using the new doesBucketExists S3 client method? https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/AmazonS3Client.html#doesBucketExistV2-java.lang.String-