Remember local ZIM library and (re-)open ZIM files easily

[practik]

I've just installed Kiwix as a Firefox add-on and am finding that my settings are not being remembered. Every time I open Kiwix, I have to reselect the ZIM archive I want to use. This is even true if I open Kiwix in one tab, load the ZIM archive there, and then open Kiwix again in a second tab.

I also tried switching the content injection mode from JQuery to ServiceWorker, and the next time I opened Kiwix, it had defaulted back to JQuery.

Fortunately, ZIM archive and content injection mode are the only two settings there are right now, and once the ZIM archive is (re)loaded, Kiwix works fine. Still, having to load it every time is a bit annoying.

(Kiwix 2.2.0 on Firefox 58.0.2 on macOS 10.12)

[Jaifroid]

The problem here is that HTML apps are sandboxed in browsers and don't have any access to the file system unless the user explicitly picks the file in question. There has been some work on removing the need for the file picker using XHR Range Requests (#292), but currently it doesn't work properly on Firefox.

Regarding content injection mode -- this value is stored in a cookie as "lastContentInjectionMode" and should be reloaded, so this may be a bug.

[practik]

Thanks, Jaifroid.

HTML apps are sandboxed in browsers and don't have any access to the file system unless the user explicitly picks the file in question

Just to be sure we understand each other correctly: Right now I have to select the file every time I open Kiwix. Are you saying that that's the currently expected behavior? I wasn't surprised that I had to pick the file the first time I opened Kiwix, but I didn't expect I would have to do it every time.

Regarding content injection mode -- this value is stored in a cookie

I'm not seeing that cookie in Firefox's built-in cookie viewer (under about:preferences#privacy). Should I be seeing it there?

[Jaifroid]

@practik I'm not really sure about the browser extension, but when using in the browser launched from a location like file:///C:/kiwix-js/www/index.html in a browser then yes, unfortunately you do currently have to pick the ZIM file every time for kiwix-js to be able to access the file. There's no way of remembering the file access granted by the user in a browser context.

In an app context, such as the Firefox OS app, or the Kiwix JS Windows app, all based on Kiwix JS, you don't have to pick the ZIM file each time. As I said, @sharun-s is working on a method that might allow XHR range requests in browser context.

Regarding the cookie, we need to try to reproduce the issue to see if it's a bug in the code. I haven't worked on the Firefox add-on, and don't know if serviceworker is truly available in add-on context. I'll let @mossroy answer on that one...

[mossroy]

In general, browsers are very careful about security. They need to make sure that a random website can not read your local files in your back. For that reason, there is not standard javascript API that allows to directly read a file on the user's filesystem. However, if the user manually selects a file (through a drag-and-drop or file input button), this file can be read and used in javascript (as long as the window/tab is not closed). That's how kiwix-js works. So, even if we remembered the file location, we would not be able to re-open the file, because there's no standard API for that. There is "filesystem" API (only on Chrome : https://www.html5rocks.com/en/tutorials/file/filesystem/), but it only allows to read/write files in an emulated sandboxed filesystem, not on the actual filesystem of the user.

Now, when this javascript code does not come from a random website but from a more "secure" location, the browser engine might give us more technical possibilities. It was the case on Firefox OS, with the "DeviceStorage" API. It is the case on the Windows UWP app, with a specific API. It is unfortunately not the case in browser extensions, at least for now. There are discussions about that, but no real plans for an implementation. See https://bugzilla.mozilla.org/show_bug.cgi?id=1246236

What @sharun-s worked on is a kind of workaround : instead of trying to read a file from the user's filesystem, we might provide the ZIM file(s) with the kiwix-js code. In this case, the browser does not consider it's a security issue. But there are several drawbacks. In particular, it means that you need to deploy a different browser extension for each ZIM file. It can be a good idea for the most popular ones (concept of "custom apps"), but is not possible for all of them. And the ZIM files can not be too big. See #292

Regarding the "content injection" mode, the last used value is stored in a cookie, and read on next opening. I does seem to work. But what is confusing is the fact the Mozilla does not allow ServiceWorkers inside extensions (see https://bugzilla.mozilla.org/show_bug.cgi?id=1344561) : you might have noticed that the status stays in orange as "not registered". So, even if you clicked on it, it has not been activated, and not stored in the cookie. That's why this setting is not restored afterwards. If you try in the Chrome extension, you'll see it working (because Google allows ServiceWorkers inside their extensions)

[practik]

Thank you for that very detailed explanation, @mossroy.

This may be entirely irrelevant, but I know that the developer of Greasemonkey has been grappling with local file access as well – there's some discussion here: https://github.com/greasemonkey/greasemonkey/issues/2612

[kelson42]

@mossroy In a total unrelated repo, we have tested PouchDB (a pure client side js db library) for Gutenberg scraper https://github.com/openzim/gutenberg/issues/91. I wonder is this (with indexdb) could be used here to store local settings and library.

[Jaifroid]

@kelson42 We don't have a problem storing local settings. Cookies work, and localStorage can be used in contexts that don't allow cookies (only Electron in some configurations). The problem that was raised in this issue was that the user must pick the ZIM file each time the app is started in browser and browser-extension contexts. This is a security limitation of all browsers. In app contexts, there is no such limitation (hence there is no need to re-pick the ZIM in Electron, NWJS or UWP apps based on Kiwix JS).

[kelson42]

@Jaifroid Thx, I have changed the title which is partly missleading. I will also hide our last comments as they are out of topic.

[Jaifroid]

This issue is still not achievable in extensions other than by using the File System Access API in Chromium browsers, for which see #656.