s3-multipart
s3-multipart copied to clipboard
Utilities to do parallel upload/download with Amazon S3
I am calling the download script as : python s3_mp_download.py S3://xxxxxxxx/backup_20170324.tar backup.tar It runs for a while and error outs later saying : ERROR:s3-mp-download:do_part_download() takes exactly 1 argument (9 given)
when i am uploading a file from java getting error .com.amazonaws.services.s3.model.AmazonS3Exception: Your proposed upload is smaller than the minimum allowed size (Service: Amazon S3; Status Code: 400; Error Code: EntityTooSmall;...
Hey there, In s3-multipart-download.py you repeat the `connect_s3()` call https://github.com/mumrah/s3-multipart/blob/master/s3-mp-download.py#L110-L113 which causes this to happen: ``` DEBUG:s3-mp-download:CLI args: Namespace(dest='./test', force=False, max_tries=5, num_processes=2, quiet=False, secure=True, split=32, src='s3://cgl-driver-projects/test/test', verbose=True) DEBUG:s3-mp-download:split_rs: SplitResult(scheme='s3', netloc='cgl-driver-projects',...
There seems to be a bug when retrying download. Currently it tries to call do_part_download and get following error: ``` ERROR:s3-mp-download:do_part_download() takes exactly 1 argument (9 given) ``` The PR...
Adds the ability to download multiple files based on a specified prefix. Creates identical directory tree at the destination. Example: s3-mp-download.py -v s3://bucket-name/path-to-bucket/prefix /tmp/path-to-local-dir Will download all objects contained in...
This line caused error `ERROR:s3-mp-download:do_part_download() takes exactly 1 argument (9 given)`
(1), multi part upload progress not display in function as below. can display when use print def progress(x,y): logger.debug("Part %d: %0.2f%%" % (i+1, 100.*x/y)) (2), and is it possible to...
This brings in some fixes related to retry logic and connection cleanup
- code passes "flake8 --ignore E501 ." Largely automated cleanups using pycharm to update the code to PEP8 style. One logic bug was identified in s3-mb-update.py - the retry logic...
This patch changes formatting to make it PEP-8 compliant.