leofs icon indicating copy to clipboard operation
leofs copied to clipboard

[leo_gateway] Support GET Bucket (List Objects) Version 2

Open mocchira opened this issue 6 years ago • 0 comments

AWS has published the new API - GET Bucket (List Objects) Version 2 (https://docs.aws.amazon.com/AmazonS3/latest/API/v2-RESTBucketGET.html) as a successor of Version 1 (https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGET.html).

With this new I/F, we can implement listing objects more efficiently especially in case the key count goes beyond max-keys because the new one abolished NextMarker storing a raw key and instead introduced a new field NextContinuationToken storing a opaque value which the S3 implementer only understand so that means the new implementation doesn't need to fetch NextMarker in the prefix match way on the subsequent requests but is able to fetch the next page in more efficient ways with a opaque value like the opaque value can be transformed into the offset where the next object resides so the system resources which were previously needed to search the next offset will no longer be needed.

mocchira avatar Sep 06 '18 08:09 mocchira