Unconfirmed: sites with >1000 objects fail to delete

[drewbo]

User Story

As a Federalist user, I want my site to be deleted when I click delete.

Background (Optional)

We had a support case two weeks ago where a user reported not being able to recreate a site after previously deleting it. The error shown in the event logs (The service instance name is taken) was caused because the previous bucket was not deleted properly (cf services reporting delete failed and more specifically Service broker error: BucketNotEmpty: The bucket you tried to delete is not empty). The support ticket was resolved by manually removing all bucket items and deleting the bucket. We noted that at the time of deletion, the bucket contained more than 1400 items.

It is possible (though unconfirmed) that the delete failed because the relevant S3SiteRemover code only lists the first 1000 keys for deletion per the SDK documentation. This portion of the code needs to be updated to list all keys for deletion.

Acceptance Criteria

• [ ] Optional: create a test case
• [ ] Confirm deletion of a test site with more than 1000 items on staging
• [ ] Changes deployed to production

After evaluating, edit this part:

Level of effort - low

[drewbo]

Reading further, the large site deletion method uses a helper method, also named listObjects rather than the AWS SDK method. This method should, in theory, be paginating the results correctly.

My next thought is that the delete failed because of selecting only certain prefixes but impossible to confirm without the bucket in question. Leaving open for future further investigation