Paginated version of S3.listObjectsV2

[julien-c]

trafficstars

I see references to a Paginator in the code but haven't found any doc on how to use it.

My use case is I want to list all objects from my S3 bucket (which has more than 1_000 keys)

Thank you!

[julien-c]

Any news? Thanks :)

[abhilash1in]

This is my guess:

If data is paginated, results of the S3.listObjectsV2(params, callback) call will have IsTruncated set to true and will have a value in NextContinuationToken. For your next call to S3.listObjectsV2(params, callback), set the ContinuationToken property of the params object to the NextContinuationToken value from the previous token. You need to keep doing this recursively until IsTruncated is false.

Refer to documentation for more info on the properties of request params and response of S3.listObjectsV2(params, callback) .

[julien-c]

Yes, @abhilash1in, that's what I am doing. But I was wondering if there was a built-in helper.

[marceloavf]

In listObjects there is a .eachPage method, seems that it's available too in listObjectsV2 but not documented, that's really confusing.

[Abhishek-Kaudare]

What is the Validity of NextContinuationToken from pagination as I tend to use it after days to restart processing

[github-actions[bot]]

Greetings! We’re closing this issue because it has been open a long time and hasn’t been updated in a while and may not be getting the attention it deserves. We encourage you to check if this is still an issue in the latest release and if you find that this is still a problem, please feel free to comment or open a new issue.

[julien-c]

unstale

[newforma-bclemans]

FWIW, this is what I put together in order to paginate through a non-versioned bucket and delete all objects. I hope this example helps.

export async function deleteBucketObjects(BucketName, region) {
    const client = new S3Client({region})

        for await (const data of paginateListObjectsV2({client, pageSize: 1000}, {Bucket: BucketName})) {
            const s3Objects = data.Contents.map(({Key}) => ({Key}))
            const bucketParams = {Bucket: BucketName, Delete: {Objects: s3Objects}}
            try {
                console.log(data)
                  await client.send(new DeleteObjectsCommand(bucketParams))
            } catch (err) {
                console.info(`No files in S3 bucket: ${BucketName}`)
            }
        }
}

[github-actions[bot]]

Greetings! We’re closing this issue because it has been open a long time and hasn’t been updated in a while and may not be getting the attention it deserves. We encourage you to check if this is still an issue in the latest release and if you find that this is still a problem, please feel free to comment or open a new issue.