Write multiple files to a directory

[EtienneBruines]

The user would select a directory as usual:

const opts = {type: 'open-directory'};
const handle = await window.chooseFileSystemEntries(opts);

It would be very useful to allow the PWA to write multiple files into that directory (optional: an option for write-only access).

Two use-cases:

• An IDE that has to write a lot of config files (e.g. the .idea directory, or the .git directory)
• Downloading multiple files into a specific directory

The second use-case is one we're currently exploring. At the moment, we resorted to creating a ZIP-file in the browser (JSZip :cry:), and then serving it using saveAs. This results in horrible performance (about a minute per 100MB) - and requires the user to extract the archive again after downloading. It is also limited to the amount of RAM the user has. 4GB RAM cannot create an archive with 10x 1GB files.

It would be better for the user-experience if the user could select the target destination folder, and we can save the files into there directly.

---

Other thoughts:

• the user might be afraid that we "override" his favorite files in the selected directory
• perhaps a "bulk request" might be an option? The user selecting a directory, the PWA providing a list of files that will be written, and then the user receiving a prompt "The PWA wants to write the following files: A, B, C, ..." and then having to confirm.

---

off-topic:

Alternatively we explored downloading all files one-by-one using saveAs - which works fine as long as the user defaults to ~/Downloads - but as soon as the user has configured "prompt on download" - the user gets spammed for every single file. And there's no way of knowing what the user configured; meaning that if this would be an acceptable experience for 80% of users, because it's unacceptable for 20% it's a no-go overall and therefore benefits 0% of users. (Talking about 100+ files)

[jimmywarting]

off topic but i just want to share a alternativ 200 line streaming zip version with less overhead:

https://github.com/jimmywarting/StreamSaver.js/blob/master/examples/zip-stream.js view-source:https://jimmywarting.github.io/StreamSaver.js/examples/saving-multiple-files.html

JSZip creates a lot of in memory data and operates mostly on string instead of typed arrays. it also don't have a proper method for flushing data as it creates the zip file.

then there is also conflux as well with both read & write

all client zip libraries i know of (including mine & Mac built in arciver) are limited to 4gb zip file due to lack of zip64 extension

[mariusa]

+1. Our use case: a fiscal receipt printer has it's own app installed on a computer. The fiscal printer app monitors a folder for new files created and prints receipts according to the file contents.

We need to generate files in a specific folder from a web app, without the user having to "save" each file. That will happen transparently in background, on specific user actions, eg add a cash payment in the web app.

Reported to Chrome: https://bugs.chromium.org/p/chromium/issues/detail?id=1109683

[gruenich]

+1. My use case: My application downloads dozens of files from a server A via a REST API. The must be saved to a folder as regular files, because another local software B grabs these files and automatically processes them. A Zip container does not work, as I cannot change the system A and B. Further, the files would be combined several GiB of size (no further compression possible), so we prefer having dozens of them.

Similar to Etienne, I'd like to get full control over a directory, regarding reading/writing/deleting files. Maybe even creating / deleting subfolders with recursive rights.

[mariusa]

At least writing is actually possible now. With Chrome 85 (dev channel) :

           dhandle = await window.showDirectoryPicker()
           dhandle.requestPermission({ writable: true })
//write as many files as needed:
            let fhandle = await dhandle.getFileHandle('filename1', { create: true })
            const writable = await fhandle.createWritable()
            await writable.write('some content')
            await writable.close()

Issues:

1. How to persist dhandle between page reloads?

2. How to persist dhandle between browser restarts or opening/closing the tab? Chrome says it's valid until tab is closed.

[jimmywarting]

I think you should be storing the folder handle in indexed db

[mkruisselbrink]

I think everything described here should indeed already be possible. To get a writable directory handle indeed just call showDirectoryPicker followed by requestPermission (and when we fix #89 we'll be able to change that to just call showDirectoryPicker with the right options). If you want to persist the handle between page reloads/browser restarts, store the handle in IndexedDB. Currently permissions will still expire, so on reload you'd have to re-request permission. In the future we hope to auto-grant such re-requested permissions for for example installed PWAs (i.e. you'd still need to re-request permission on reload, but there wouldn't be always be a prompt).

[mariusa]

In the future we hope to auto-grant such re-requested permissions for for example installed PWAs (i.e. you'd still need to re-request permission on reload, but there wouldn't be always be a prompt)

Is there an issue / chrome bug we can follow to get updates on this?

[mkruisselbrink]

That would be https://crbug.com/1011533

[mariusa]

Another question, please:

To get a writable directory handle indeed just call showDirectoryPicker followed by requestPermission (and when we fix #89 we'll be able to change that to just call showDirectoryPicker with the right options).

But, if 'permissions will still expire, so on reload you'd have to re-request permission', that means the code will always be like this?

let dhandleo = store.get('dhandle')
dhandleo.onsuccess = function() {
    if (dhandleo.result === undefined) {
        // no record with that key
        globalThis.dhandle = await window.showDirectoryPicker({ writable: true })
       store.add(globalThis.dhandle)
    } else {
        globalThis.dhandle = dhandleo.result
        globalThis.dhandle.requestPermission({ writable: true }) //always request permission, as it's not persisted with dhandle
    }
}

Even in the installed PWA case, it will still be like above? (the difference being there's no user prompt)

[mkruisselbrink]

Yes, that is our current thinking/plan. One reason being is that it avoids websites that accidentally only work when they are an installed PWA.

[my2iu]

I would prefer a separate API explicitly for saving multiple files instead of having the user give write permission for an entire directory. Some users don't really understand how directories work, and there's going to be issues where the user will accidentally choose a directory which isn't valid for security reasons (is it valid for users to choose their entire Documents directory?). Having an API where a web page can specify a bunch of files along with suggested file names would be easier for programmers, easier for users, and is better from a security standpoint because web pages don't have to request more permissions than necessary to do their work.

[twilson7755]

I was trying to find some sort of update for when it will be possible to persist permissions for PWA's for this exact functionality. I took a look at https://crbug.com/1011533 but the last update was from July. The target is listed as 87 but given that there doesn't seem to be much activity for this issue, I was wondering if the timeline for this is going to be moved out? This functionality is critical for us to utilize these new enhanced file system API's ... without it, we will need to utilize something like Electron. We are hoping to make a decision very soon, so any sort of update would be greatly appreciated.

[mariusa]

Is there any other way to persist dhandle, besides indexedDb? localstorage doesn't work, as it needs strings and JSON.stringify(dhandle) returns {}

with indexedDb, await window.showDirectoryPicker auto-closes the transaction and can't save the actual handler.