valkyrie icon indicating copy to clipboard operation
valkyrie copied to clipboard

Fedora adapter(s) should support streaming binary content

Open escowles opened this issue 7 years ago • 3 comments

Binary content can easily be larger than memory, so streaming is a crucial feature. If Faraday doesn't support streaming (https://github.com/lostisland/faraday/pull/604), then we should fall back on Net::HTTP or something else that does for this (e.g., https://github.com/samvera/active_fedora/blob/master/lib/active_fedora/file/streaming.rb).

escowles avatar Jul 13 '17 12:07 escowles

Looks like Faraday supports streamed requests now.

tpendragon avatar Aug 02 '18 19:08 tpendragon

This ticket is complete when I can download a file that is larger than the memory of the server running the Valkyrie application

escowles avatar Aug 06 '18 17:08 escowles

upload needs to use a block to send chunks of uploads, instead of reading the file into memory with tempfile.read: https://github.com/samvera-labs/valkyrie/blob/master/lib/valkyrie/storage/fedora.rb#L39

escowles avatar Aug 10 '18 19:08 escowles