metacatui icon indicating copy to clipboard operation
metacatui copied to clipboard

Streaming download + zip approach for Partial Data Download

Open shirlysteph opened this issue 7 months ago • 0 comments

Partial Data Download Feature Improvement

Leverages streaming ZIP creation directly in the browser and stream data into the ZIP file as it downloads, rather than buffering everything first. This may avoid loading all file contents into memory before creating the ZIP archive, making download more efficient, and also increasing the downloadable file size limits.

Possible ways to do this

  1. Use fflate - fflate supports streaming compression and blob streaming, which allows you to zip files and push them directly to a download stream without storing everything in memory.
  2. Use zip-stream

**Priority: **

shirlysteph avatar Apr 21 '25 18:04 shirlysteph