Support for multi-file upload?

[rbrecheisen]

Hi, I managed to get the single file upload working. Works like a charm! Very nice :)

I was just wondering whether it would be possible to upload multiple files simultaneously? This would be quite important for my use case since I have potentially thousands of files to process. I could also ZIP them and upload that but I'm not sure how blocking the unzip process would be. Can that be done "natively" in NGINX or do I need to do that in Django itself?

Hope you have some tips!

[vsoch]

hey @rbrecheisen ! So there is no reason you couldn't try running the upload script in parallel (and that could be done within the script with multiprocessing or something as simple as the parallel tool on the command line). But that upload process would be limited by the server itself. If you have a small number of files maybe that's reasonable, but otherwise you probably want to consider some kind of storage service alongside Django that has a custom upload API. For example, you could generate signed URLs to upload to Google Cloud Storage. or spin up another container like Minio and use their S3 compatible multi part API. This is actually what I did for Singularity Registry Server here is the minio file.