s3cmd icon indicating copy to clipboard operation
s3cmd copied to clipboard

Feature request: ListObjectsV2 request option in `ls` command

Open snosratiershad opened this issue 3 years ago • 6 comments

In progress of this PR I noticed s3cmd doesn't implemented the ListObjectsV2 requests and all operations in ls command is under ListObjects.

Is there any plan to support it? let me know and I would create a PR ASAP.

snosratiershad avatar Aug 08 '22 22:08 snosratiershad

@fviard, do you think it's also should be included as a config variable like list_version? or a command prompt param like --list-objects-v2 as aws cli?

snosratiershad avatar Aug 22 '22 10:08 snosratiershad

I don't think that there is a big interest in switching to ListObjectV2 for performance or anything, and it is probably not possible to switch as S3 compatible servers might not have migrated to that yet.

But, it could be interesting to already have support for ListObjectV2 to be future proof and for a possible server that would not support ListObjectV1.

So, it would be good if you want to add the support for that in s3cmd :-)
I don't think that a cmdline parameter is useful, but we can probably have a config variable. Something like "use_list_object_v2" or "enable_list_object_v2".

fviard avatar Aug 27 '22 20:08 fviard

on it!

snosratiershad avatar Aug 28 '22 01:08 snosratiershad

@fviard Done.

snosratiershad avatar Aug 28 '22 01:08 snosratiershad

Amazon recently added Amazon S3 Express One Zone. This is a high-performance, single-Availability Zone storage class purpose-built to deliver consistent single-digit millisecond data access for your most frequently accessed data and latency-sensitive applications. S3 Express One Zone can improve data access speeds by 10x and reduce request costs by 50% compared to S3 Standard.

Amazon S3 Express One Zone doesn't support ListObjects. It supports ListObjectsV2 API. Hence S3cmd command cannot be used with S3 express one zone directory. It fails as below. It would be good if this functionality can be implemented.

# s3cmd ls -r -p s3://<S3 Express One Zone bucket>
ERROR: S3 error: 501 (NotSupported): This bucket does not support ListObjects API. Consider using ListObjectsV2 API.

showvick avatar Dec 26 '23 05:12 showvick