uplink icon indicating copy to clipboard operation
uplink copied to clipboard

Uplink should error early if the file is too large with too small part sizes

Open shaupt131 opened this issue 2 years ago • 1 comments

Issue originated from 5/12 Arch Review meeting. Meeting Notes

A fix was made to adjust multipart part size based on file size but a user can still override part size manually. If they select a part size that exceeds current limits, or the parts will be too small, they should receive an error message.

shaupt131 avatar May 13 '22 19:05 shaupt131

The linked PR works as following:

uplink-cli automatically adjusts the part size based on the length of the file handed in. If the size is not known or cannot be fechted (ie. with an input from stdin), it defaults to 64MiB part sizes. This can be adjusted at any time by setting the flag parallelism-chunk-size to the reasonable limit. @pwilloughby brought up that AWS uses a flag called expected-size to not have the user calculate the right chunk size.

Additionally, if the user prefers to upload with a special chunk/part size, the cli compares and ensures the value specified is big enough, else returns with an error hinting about how big it needs to be. If it is bigger than necessary it will use it as requested 👍

Maybe its worth discussing tossing the flag in favor of the expected-size flag.

stefanbenten avatar May 13 '22 20:05 stefanbenten