node-mongodb-s3-backup icon indicating copy to clipboard operation
node-mongodb-s3-backup copied to clipboard

Published 20 hours ago • verified publisher icon theycallmeswift

Max upload size

Open srlowe opened this issue 9 years ago • 3 comments

Hi, Could you list the max-upload size somewhere please?

Thanks

srlowe avatar Jun 30 '16 14:06 srlowe

Hey @bgSosh. Not sure if I follow? S3 might have some size limitations, but I don't think mongodump does. Are you hitting specific issues with size?

theycallmeswift avatar Jun 30 '16 18:06 theycallmeswift

Hi swift, sorry, I wrote this in a hurry (and sounds not like I intended). S3 has a single-put limit of 5GB AFAIK, and over that (up to 5TB I think) you have to use the multipart API. Not sure which this project is using. Cheers

srlowe avatar Jun 30 '16 22:06 srlowe

The underlying code uses Knox's putFile, which you can find here: https://github.com/Automattic/knox/blob/088050868e59ec9bf5bc5fc91d606c37ca0c78d5/lib/client.js#L350-384

Looks to me like a normal streaming put operation which would indicate a 5GB limit I think. Honestly unsure though. Do you have a 5gb file you can test it on?

theycallmeswift avatar Jul 01 '16 13:07 theycallmeswift