How Big are the Parts in Upload?

[kmand]

I've noticed that uploading 40 large files (ie archives) has generated 8000 requests, apparently because glacier-cmd does a multipart upload. How big are the parts, and is this settable?

[wvmarle]

Yes, it's doing a multi-part upload; part size is at least 1 MB, size increased to have no more than 10,000 parts per archive (Amazon Glacier limit). In your case I suppose they're 1 MB chunks.

Smaller part size mean more requests (little slowdown as each request takes time to set up), but if an upload fails, less data has to be sent again. Also the status bar will update more frequent, as it's updated every time a part is uploaded.

To set it to a different size (max 4096 MB - must be in MB and a power of 2, so 1, 2, 4, 8, 16, etc) use the --partsize size option when uploading. Note that glacier-cmd will override this option if the size you set is too small, i.e. it would require >10,000 parts to upload an archive.