skbn icon indicating copy to clipboard operation
skbn copied to clipboard

Published 20 hours ago • verified publisher icon maorfr

Fails when filesize is too large

Open ButlerJoseph opened this issue 4 years ago • 1 comments

I am using the Cassandra Cain tool (https://github.com/maorfr/cain) to backup a Cassandra datacenter and it is failing when backing up files which are 500 GB in size.

Having looked at the code for this project, I've noticed that this project is using the default UploadPartSize of 5 megabytes (https://github.com/aws/aws-sdk-go/blob/fe72a52350a8962175bb71c531ec9724ce48abd8/service/s3/s3manager/upload.go#L26), this gives us the ability to files up to 52 GB in size.

Please can you consider increasing this value to 64 bytes so SKBN is able to upload files up to 671 GB in size?

ButlerJoseph avatar Nov 13 '19 16:11 ButlerJoseph

Hi @ButlerJoseph When you say it is failing - did the process fail with an error message or just hangs? What error did you see?

rrusmana avatar May 18 '20 06:05 rrusmana