txaws icon indicating copy to clipboard operation
txaws copied to clipboard

Use cooperator for hashing

Open mithrandi opened this issue 8 years ago • 0 comments

For medium sized objects (let's say larger than 1MB but smaller than 100MB), fitting the object data into memory all at once isn't a big deal, but blocking to calculate the hash will take long enough to significantly impact other things going on in the process, especially if multiple uploads are attempted at once. One possible easy fix for this is to feed data into the hash function in multiple chunks, with cooperator driving the process to allow other servicing other events at the same time.

mithrandi avatar May 05 '17 05:05 mithrandi