s3-disk-usage icon indicating copy to clipboard operation
s3-disk-usage copied to clipboard

Count space utilized by unaborted multi-part downloads

Open sumitag opened this issue 5 years ago • 2 comments
trafficstars

If you have a s3 bucket used for uploading large objects then you are most likely using multi-part upload and s3 sync automatically uses multi-part upload.

These multi-part uploads can be left open and continue to use space.

sumitag avatar Jun 14 '20 10:06 sumitag

That had not occurred to me!

Can you provide me with any sample commands to cause a multi-part upload to happen? I'd love to test this and see if I can build in support to my tool.

Thanks!

-- Doug

dmuth avatar Jun 23 '20 00:06 dmuth

aws s3api list-multipart-uploads --bucket $BUCKETNAME \
| jq -r '.Uploads[] | "--key \"\(.Key)\" --upload-id \(.UploadId)"' \
| while read -r line; do
    eval "aws s3api abort-multipart-upload --bucket $BUCKETNAME $line";
done

This is the command I used for aborting the multipart uploads

sumitag avatar Jul 04 '20 15:07 sumitag