StreamSaver.js icon indicating copy to clipboard operation
StreamSaver.js copied to clipboard

data is not appended to file until stream is closed

Open baptx opened this issue 4 years ago • 3 comments

Hello, I tested the example write-slowly.html on my Apache server with Firefox (version 84.0 64-bit from the deb package of Lubuntu 20.04) and after clicking the start button and waiting for more than 10 seconds, I can only see 2 a characters in the partially downloaded file sample.txt.part. I have to refresh the page or close the tab to see all the data downloaded.

Is there a way to fix this issue? I would like to save data in realtime so I can start playing a video saved with MediaRecorder while it is being downloaded.

baptx avatar Jan 04 '21 13:01 baptx

In fact the issue seems to be related to this one: https://github.com/jimmywarting/StreamSaver.js/issues/58 It could be useful to mention in README.md that several kilobytes of data have to be downloaded first before being appended to the file. Is there a workaround to save each byte in realtime? So we could for example read text while it is typed by someone and downloaded.

By the way, the README.md file mentions the FileSystem API but what is the difference with StreamSaver.js? Can't we append data to a file using FileSystem API (https://developer.mozilla.org/en-US/docs/Web/API/FileSystem)? This could be clarified in README.md also so we can understand the difference.

baptx avatar Jan 04 '21 13:01 baptx

So we could for example read text while it is typed by someone and downloaded.

I have manage to do it with a code editor, it must have support for file modification watch event... don't think you can see the changes happening in realtime with every program. Writing a partial movie and play it with VLC could potentially work. (depending on codec and a bounch of other stuff)


the filesystem you linked to on MDN is the "old" sandboxed filesystem (by chrome) that can store files in the browser just like indexeddb or localStorage.

The filesystem i'm referring to is the new experimental Native Filesystem where you can request a place of the user choice to read/write data, to/from files and folders

It's very different from what StreamSaver dose.

  • StreamSaver basically creates a ReadableStream that is wrapped around a local network request (using service worker) to simulate how you download files from a server. You can write data to it at your own phase... chunk by chunk instead of writing all at once
  • With Native FileSystem you can get write (and read) access to the users chosen location and re-read/write multiple times. it also have the possibility to ask in which format (extension) a user wants to save with. So you get a file handle beforehand with the chosen file name and then you write data to it. All doe you won't get the same native browser UI for downloads (no progress bar, download speed, time left, download history) all of this is now controlled by the developer.

read the EXPLAINER to understand more about the new filesystem

I bet it could potentially make StreamSaver, fileSaver and other alike a thing of the past, but not right now...


I wrote a adapter for the new filesystem so it can be somewhat polyfilled and used with other storage location such as indexeddb and memory and a limited set of native features. the syntax is the same as native filesystem so a switch from shim to native is a easy transition if it's available.

I have sometimes plan to write a adapter for NodeJS also with the same API syntax...

jimmywarting avatar Jan 04 '21 14:01 jimmywarting

So we could for example read text while it is typed by someone and downloaded.

I have manage to do it with a code editor, it must have support for file modification watch event...

@jimmywarting it could also be done without a code editor, using command line like this: for ((i=0;i<60;++i)); do cat sample.txt.part; echo; sleep 1; clear; done. However I cannot see bytes saved in real time, I have to wait until the file has several kilobytes. Is there a workaround for this?

Another way to do it would be using Selenium WebDriver and saving data using Node.js, with the writeFile / appendFile function.

baptx avatar Jan 04 '21 16:01 baptx