Enable support for service workers for new replay system

[ikreymer]

Since kiwix-desktop use a custom scheme, to enable support for the service-worker based replay, the following flag should be enabled:

scheme.setFlags(QWebEngineUrlScheme::ServiceWorkersAllowed);

https://doc.qt.io/qt-5/qwebengineurlscheme.html

Hopefully this will make it work, though there is a chance that it may not, and regular http://localhost server may be needed..

Here is an example ZIM file that works with kiwix-serve in the browser, and will hopefully work with service workers enabled zimreplay.zim (rename .zip -> .zim as github doesn't allow .zim attachments)

Here's how it looks like with kiwix serve (replay of a twitter.com/netpreserve using a baked in warc file from: https://github.com/webrecorder/replayweb.page/blob/master/examples/netpreserve-twitter.warc)

[kelson42]

@mgautierfr I have tried the simple proposal of @ikreymer but the rendering still fails. Your help is needed. It is important we secure this can work with Kiwix Desktop for Zimit.

[ikreymer]

i've spent a bit more time trying to get service workers to work in kiwix-desktop. First, would definitely need a later Qt, as per kiwix/kiwix-build#403 as the chromium that ships with default ubuntu bionic packages (5.9) is far too old. (I tried with 5.14)

Unfortunately, no matter which flags I tried, service workers did not work with the custom zim:// protocol handler. I am not too surprised, as this has been an issue with Electron for years: https://github.com/electron/electron/issues/9663 -- it's probably a Chromium issue.

But, fortunately, service workers do work when running from localhost in QtWebView, so probably the solution that will work is to run kiwix-serve inside kiwix-desktop and load the zim data via http that way, instead of via the custom protocol handler.

It should look the same/make no difference to the user, but I imagine will require some changes here, to load from http://localhost:<port> instead of from zim:// (I think it will probably be a similar solution for the kiwix-apple setup).

Here's an example of the replay within kiwix-desktop with a manually hacked localhost server:

[kelson42]

@ikreymer Thank you very much for this testing. Your are 100% sure there is nothing in your code or JS code you rely one where the protocol http:// is hardcoded?

[ikreymer]

Yes, there's nothing specific to http. It runs on https too of course. The system requires fetch and service worker support, though. Here's an even simpler zim designed to test service worker functionality: basicsw.zim (just rename .zip -> .zim as gh doesn't allow .zim ext)

If it works, it looks like this:

if service workers don't work, it looks like this:

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[ikreymer]

FWIW, Electron finally fixed support for service workers with custom protocols, so perhaps this may be more feasible as well.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[kelson42]

For people interested in a workaround for sole usage on local computer:

• Start the HTTP server an use IP 127.0.0.1 (localhost)
• Access the content via your webbrowser, the ServiceWorker based ZIM file should be perfectly readable.

[kelson42]

For people interested by this ticket, this is meanwhile on the very top of priority list, so should be fixed in the few next months.

[kelson42]

It seems that we will be able to create ZIM files without SW with Zimit in version 2.0. Therefore this feature request priority is significantly lowered.

[crowlsyong]

Glad to see this is an active thread. I am tinkering with Kiwix and I just tried making an offline website but I ran into this message when trying to access the site via a kiwix server: "This page must be loaded via an HTTPS URL to support service workers.

Try Loading HTTPS URL?" but instead of example.com it was the website I downloaded. I am a noob with Kiwix so if I'm missing something, let me know. If not, I'm excited that Zimit version 2.0 will address these issues....is there an estimate on when a noob like me will be able to update kiwix (or something like that) and begin accessing my offline sites?

❣️ Thank you devs you make me happy. ❣️

[kelson42]

@crowlsyong The only workaround for the moment for you is to start the http server on the 127.0.0.1 and you shoukd be able to see your ZIM content.

[kelson42]

Closing this ussue as Zimit2 does not rely on SW.