socketio-file-upload icon indicating copy to clipboard operation
socketio-file-upload copied to clipboard

I can't set the quantity of files that can be uploaded per time

Open Tubaleviao opened this issue 8 years ago • 3 comments

When I select something around 20 files, it send chunks for all the files. So if I lose internet connection, all the upload progress is aborted and I'll get no complete files in the server.

Something like this could be implemented? Setting, per example, 2 files per time (for sending chunks). So when one of these 2 files is fully uploaded, the system start sending chunks for the next file.

Tubaleviao avatar Jul 14 '16 18:07 Tubaleviao

+1 Previously, I built solutions using Dropzone.js as well as the "dropzone-react" component. Both offer a property that indicates how many simultaneous files to upload at a time. Seems 2 to 4 is a common preference. This is probably based on common bandwidth optimization and to ensure more files are uploaded successfully. As @Tubaleviao points out, consider a user who drops a nested folder structure with 1,000 files. It is better to stream a few files at a time than start all 1,000 streaming at one time.

Is this something that can and should be addressed in siofu?

Disclaimer, I may not fully understand what is happening behind the scenes!

TroyWolf avatar Dec 05 '17 18:12 TroyWolf

I just implemented this myself "in front" of siofu by creating a "files queue" array. When my user drops or otherwise adds files to be uploaded, instead of doing an siofu.submitFiles(), I push them onto my queue. I have code that monitors the queue and submits an array of 2 files at a time. It waits for them to complete before adding up to 2 more. Repeat until the queue is empty. I want to improve it so that I always have 2 in flight. Right now, it processes 2 at a time, but if one finishes first, my code waits for the 2nd to also complete before I submit 2 more.

I wanted to reply to say it is possible to hack together a queue yourself.

Worth mentioning that in creating this solution, I found myself wishing there was a "global" siofu method I could call to know if any uploads are still in flight. I ended up maintaining 2 counters: started and completed. When I submit 2 files, for example, I add 2 to my 'started' counter. In my siofu completed handler, I add 1 to my 'completed' counter. So while completed < started, I know there are still uploads in progress.

TroyWolf avatar Dec 06 '17 00:12 TroyWolf

No one could help me with this, so I developed my own uploader up.io. I don't know if it's faster than the siofu, but I'll test this later.

Feel free to use it if you want, I hope it can help you \o/

Tubaleviao avatar Mar 07 '18 06:03 Tubaleviao