blazer
blazer copied to clipboard
kurin
Append the hash by default.
By default, blazer will do a pass over the data to compute the hash so that it can be sent at the start of the upload request, as described in https://www.backblaze.com/b2/docs/uploading.html
Blazer can either do this in memory, which is the default, or can copy the input into a temp file, or it can just make two passes over the input if the input supports seeking. Each of these has side effects that might not be obvious to a user (e.g., if you're using blazer to copy files out of NFS, blazer will happily read every file twice).
Blazer should append the hash at the end by default. This is not recommended by blazer, but if there's a functional difference between the two options it's not obvious to me what it is.
Blazer should append the hash at the end by default. This is not recommended by blazer
Why isn't it recommended?
B2 doesn't support chunked transfers
What advantages would chunked transfers provide?
I mistyped, I meant it's not recommended by Backblaze. I don't know why, and I still haven't thought of anything.
It's been a while, but I think the reason I wanted chunked transfers was so that I wouldn't have to set the content-length header on upload. For byte streams (which is my use case) I'd still have to buffer the whole file somewhere, so I didn't develop this.
Do you have a use-case for it?
it can just make two passes over the input if the input supports seeking
btw this is no longer true, it only makes one pass in this instance.
Oh, here's a good reason: if the file changes during upload, appending the hash will happily submit the corrupted file, but a two-pass solution would have the upload be rejected as the hashes wouldn't match.