uplink
uplink copied to clipboard
Uplink should error early if the file is too large with too small part sizes
Issue originated from 5/12 Arch Review meeting. Meeting Notes
A fix was made to adjust multipart part size based on file size but a user can still override part size manually. If they select a part size that exceeds current limits, or the parts will be too small, they should receive an error message.
The linked PR works as following:
uplink-cli automatically adjusts the part size based on the length of the file handed in. If the size is not known or cannot be fechted (ie. with an input from stdin), it defaults to 64MiB part sizes. This can be adjusted at any time by setting the flag parallelism-chunk-size
to the reasonable limit. @pwilloughby brought up that AWS uses a flag called expected-size
to not have the user calculate the right chunk size.
Additionally, if the user prefers to upload with a special chunk/part size, the cli compares and ensures the value specified is big enough, else returns with an error hinting about how big it needs to be. If it is bigger than necessary it will use it as requested 👍
Maybe its worth discussing tossing the flag in favor of the expected-size flag.