metacatui
metacatui copied to clipboard
Streaming download + zip approach for Partial Data Download
Partial Data Download Feature Improvement
Leverages streaming ZIP creation directly in the browser and stream data into the ZIP file as it downloads, rather than buffering everything first. This may avoid loading all file contents into memory before creating the ZIP archive, making download more efficient, and also increasing the downloadable file size limits.
Possible ways to do this
- Use fflate - fflate supports streaming compression and blob streaming, which allows you to zip files and push them directly to a download stream without storing everything in memory.
- Use zip-stream
**Priority: **