alluxio icon indicating copy to clipboard operation
alluxio copied to clipboard

[WIP] Add support for non-backslash delimiters

Open ZhuTopher opened this issue 2 years ago • 3 comments

What changes are proposed in this pull request?

This PR changes the behaviour of the S3 REST API to no-longer throw "Alluxio S3 API only support / as delimiter." when calling ListObjects(V2) with delimiter set to values other than /.

  • For context, see this comment thread on this PR: https://github.com/Alluxio/alluxio/pull/15333#discussion_r854621242

Also, there is a new property key alluxio.proxy.s3.list.objects.return.folders to control how to return folders in the response of ListBucketResult:

ListObjects(V2) has disparity on how to handle “folders” created in S3:

  • Folders created via AWS S3 web UI are included in response with size 0
    • We are currently using this implementation (i.e: alluxio.proxy.s3.list.objects.return.folders=true by default
  • “Logical” folders created as parent paths to objects (i.e: dir/ from PUT bucket/dir/object) are not included

Why are the changes needed?

Improves conformity with the S3 API.

Does this PR introduce any user facing changes?

New property key alluxio.proxy.s3.list.objects.return.folders.

ZhuTopher avatar Apr 22 '22 00:04 ZhuTopher