alluxio
alluxio copied to clipboard
[WIP] Add support for non-backslash delimiters
What changes are proposed in this pull request?
This PR changes the behaviour of the S3 REST API to no-longer throw "Alluxio S3 API only support / as delimiter."
when calling ListObjects(V2) with delimiter
set to values other than /
.
- For context, see this comment thread on this PR: https://github.com/Alluxio/alluxio/pull/15333#discussion_r854621242
Also, there is a new property key alluxio.proxy.s3.list.objects.return.folders
to control how to return folders in the response of ListBucketResult
:
ListObjects(V2) has disparity on how to handle “folders” created in S3:
- Folders created via AWS S3 web UI are included in response with size 0
- We are currently using this implementation (i.e:
alluxio.proxy.s3.list.objects.return.folders=true
by default
- We are currently using this implementation (i.e:
- “Logical” folders created as parent paths to objects (i.e:
dir/
fromPUT bucket/dir/object
) are not included
Why are the changes needed?
Improves conformity with the S3 API.
Does this PR introduce any user facing changes?
New property key alluxio.proxy.s3.list.objects.return.folders
.