uppy Resumable uploads with S3 Multipart

Resumable uploads with S3 Multipart

Open stoopman opened this issue 4 years ago • 7 comments

I choose Uppy because of it's feature to resume large file uploads. TUS and Golden Retriever look nice, but since we are dealing with large files (of 100MB+ up to a few GB) we chose to use toe direct S3 Multipart upload implementation in Uppy which works together with Companion.

Unfortunately this setup does not work with resuming file uploads after a browser refresh. All multipart chunks are being re-uploaded every time.

It is possible to add support for this in the S3 Multipart plugin or is the alternative switching to TUS uploads and creating a TUS server that uploads to S3? (For instance using https://github.com/ankitpokhrel/tus-php). I get the feeling that this will slow uploads down, as you don't have a direct upload connection to S3 this way.

Thanks! Rick

Mar 11 '20 12:03 stoopman

The intended path for S3 resumable uploads would be Golden Retriever + S3 Multipart, but that probably doesn't do the trick right now. We need to add stuff to the S3 Multipart plugin so Golden Retriever knows which data it needs to store and load, and for large files we still need to work on #1634, which would definitely take a while :(

Tus has it builtin through tus-js-client, but that is more a happy accident from Uppy's POV :)

Mar 30 '20 11:03 goto-bus-stop

Is this in pipeline? Any idea on when this feature will be implemented?

Dec 01 '20 13:12 IndrajaPunna

Hi there, It looks like golden retriever now supports ghosts!

Is there a way to use Golden Retriever with S3 multipart now? When I try the uploads show(and are not paused or errored), however they never progress and sit at whatever percentage they were at before.