s5cmd icon indicating copy to clipboard operation
s5cmd copied to clipboard

Published 20 hours ago • verified publisher icon peak

Checksum verification

Open boraberke opened this issue 2 years ago • 2 comments

boraberke avatar Aug 19 '22 15:08 boraberke

Below are the results of a benchmark test of small files. With checksum verification, it slows down around %50.

Scenario Summary
upload small files 'PR:478' ran 1.69 ± 0.41 times faster than 'PR:491'
download small files 'PR:478' ran 1.56 ± 0.05 times faster than 'PR:491'

boraberke avatar Aug 26 '22 06:08 boraberke

This PR is still in draft as it is depending on migrating to aws-sdk-go-v2. Even if migration to v2 happens, this implementation can still stay under discussion as it slows down the performance of s5cmd by at least 50%.

boraberke avatar Sep 16 '22 12:09 boraberke

Closing this PR since we have decided not to migrate to aws-sdk-go-v2. Thanks @boraberke

sonmezonur avatar Mar 27 '23 07:03 sonmezonur