openzim
`sw.js` has not been extracted from the ZIM by the time `load.js` tries to register it.
See https://github.com/kiwix/kiwix-android/pull/3580 for full details.
While testing with Zimit-based ZIMs I have a bug. Sometimes it shows infinitely loading and sometimes it shows Sorry this URL was not found in the archive but if I am again loading the same zim file or page it is working.
https://github.com/kiwix/kiwix-android/assets/34593983/de83490a-9959-4990-a06a-7f1cdc3a5ad9
https://github.com/kiwix/kiwix-android/assets/34593983/e00d8c54-37e1-4245-921b-7a59c05e161a
Whenever this bug occurs the logs show an undefined inside the URL that is returned by the libkiwix.
Could not get Item for URL = https://kiwix.app/A/undefinedH/journals.openedition.org%2Fbibnum%2F
original exception = java.lang.Exception: Cannot find entry
2023-12-07 11:54:20.553 9226-9279 ZimFileReader org.kiwix.kiwixmobile D getting mimetype for https://kiwix.app/A/undefinedH/journals.openedition.org%2Fbibnum%2F = null
2023-12-07 11:54:20.554 9226-9724 ZimFileReader org.kiwix.kiwixmobile E Could not get Item for url = https://kiwix.app/A/undefinedH/journals.openedition.org%2Fbibnum%2F
original exception = java.lang.Exception: Cannot find entry
@mgautierfr Confirms that this undefined is added in the JS code see Originally posted by @mgautierfr in https://github.com/kiwix/kiwix-android/issues/3580#issuecomment-1846931494.
In Android ServiceWorker is added to the webview initially when the application is launched, and this worker is working correctly as it is running while loading the ZIM files.
https://github.com/kiwix/kiwix-android/blob/78e186d65f200ecedc52c78e928da7f199feb8e6/core/src/main/java/org/kiwix/kiwixmobile/core/ServiceWorkerInitialiser.kt#L32
So we suspect the issue might be in load.js (Probably it takes time to register). For more details, you can read the comments from https://github.com/kiwix/kiwix-android/issues/3580#issuecomment-1844925776
Please improve this ticket's description. The title makes little sense: extracting file from the ZIM is not the responsibility of this scraper…
@rgaudin I have updated the ticket's description. If you want anything else from me let me know.
Since it's extracted from the current URL, it would be good to have its value(s)
https://github.com/openzim/warc2zim/blob/main/src/warc2zim/templates/load.js#L24
@rgaudin I have analyzed the behavior this error comes when "ww init" calls twice in sw.js, so is there any condition where this code is executed twice e.g. waiting for the service worker or something else?
"ww init", source: https://kiwix.app/A/sw.js (17)
[INFO:CONSOLE(17)] "ww init", source: https://kiwix.app/A/sw.js (17)
2023-12-14 14:16:58.537 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(9)] "Uncaught TypeError: t.waitUntil is not a function", source: https://kiwix.app/A/sw.js (9)
2023-12-14 14:16:58.538 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(9)] "Uncaught TypeError: t.waitUntil is not a function", source: https://kiwix.app/A/sw.js (9)
2023-12-14 14:16:58.556 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(42)] "final: https://kiwix.app/A/journals.openedition.org/bibnum/", source: https://kiwix.app/A/load.js (42)
2023-12-14 14:16:58.564 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(17)] "ww init", source: https://kiwix.app/A/sw.js (17)
2023-12-14 14:16:58.565 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(9)] "Uncaught TypeError: t.waitUntil is not a function", source: https://kiwix.app/A/sw.js (9)
2023-12-14 14:16:58.567 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(9)] "Uncaught TypeError: t.waitUntil is not a function", source: https://kiwix.app/A/sw.js (9)
2023-12-14 14:16:58.584 18592-18802 ZimFileReader org.kiwix.kiwixmobile D getting mimetype for https://kiwix.app/A/topFrame.html = text/html
2023-12-14 14:16:58.656 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(42)] "final: https://kiwix.app/A/journals.openedition.org/bibnum/", source: https://kiwix.app/A/load.js (42)
2023-12-14 14:16:58.683 18592-18802 ZimFileReader org.kiwix.kiwixmobile D getting mimetype for https://kiwix.app/A/topFrame.html = text/html
2023-12-14 14:16:58.837 18592-18802 ZimFileReader org.kiwix.kiwixmobile E Could not get Item for url = https://kiwix.app/A/undefinedH/journals.openedition.org%2Fbibnum%2F
original exception = java.lang.Exception: Cannot find entry
2023-12-14 14:16:58.837 18592-18802 ZimFileReader org.kiwix.kiwixmobile D getting mimetype for https://kiwix.app/A/undefinedH/journals.openedition.org%2Fbibnum%2F = null
2023-12-14 14:16:58.837 18592-19216 ZimFileReader org.kiwix.kiwixmobile E Could not get Item for url = https://kiwix.app/A/undefinedH/journals.openedition.org%2Fbibnum%2F
original exception = java.lang.Exception: Cannot find entry
2023-12-14 14:16:58.854 18592-18968 ZimFileReader org.kiwix.kiwixmobile E Could not get Item for url = https://kiwix.app/A/undefinedH/journals.openedition.org%2Fbibnum%2F
original exception = java.lang.Exception: Cannot find entry
2023-12-14 14:16:58.855 18592-18802 ZimFileReader org.kiwix.kiwixmobile E Could not get Item for url = https://kiwix.app/A/undefinedH/journals.openedition.org%2Fbibnum%2F
original exception = java.lang.Exception: Cannot find entry
2023-12-14 14:16:58.855 18592-18802 ZimFileReader org.kiwix.kiwixmobile D getting mimetype for https://kiwix.app/A/undefinedH/journals.openedition.org%2Fbibnum%2F = null
2023-12-14 14:16:58.933 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(121)] "TypeError: Cannot read properties of null (reading 'warcType')", source: https://kiwix.app/A/sw.js?replayPrefix=&root=kiwixapp (121)
2023-12-14 14:16:58.934 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(121)] "Ignoring headers, error parsing headers response for: https://journals.openedition.org/bibnum/", source: https://kiwix.app/A/sw.js?replayPrefix=&root=kiwixapp (121)
2023-12-14 14:16:58.934 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(0)] "The FetchEvent for "https://kiwix.app/A/mp_/https://journals.openedition.org/bibnum/" resulted in a network error response: the promise was rejected.", source: (0)
2023-12-14 14:16:58.934 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(121)] "TypeError: Cannot read properties of null (reading 'warcType')", source: https://kiwix.app/A/sw.js?replayPrefix=&root=kiwixapp (121)
2023-12-14 14:16:58.934 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(121)] "Ignoring headers, error parsing headers response for: https://journals.openedition.org/bibnum/", source: https://kiwix.app/A/sw.js?replayPrefix=&root=kiwixapp (121)
2023-12-14 14:16:58.934 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(0)] "The FetchEvent for "https://kiwix.app/A/mp_/https://journals.openedition.org/bibnum/" resulted in a network error response: the promise was rejected.", source: (0)
2023-12-14 14:16:58.934 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(25)] "Uncaught (in promise) RangeError: Failed to construct 'Response': The status provided (0) is outside the range [200, 599].", source: https://kiwix.app/A/sw.js?replayPrefix=&root=kiwixapp (25)
2023-12-14 14:16:58.934 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(25)] "Uncaught (in promise) RangeError: Failed to construct 'Response': The status provided (0) is outside the range [200, 599].", source: https://kiwix.app/A/sw.js?replayPrefix=&root=kiwixapp (25)
[INFO:CONSOLE(17)] "ww init", source: https://kiwix.app/A/sw.js (17)
2023-12-14 14:30:31.531 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(9)] "Uncaught TypeError: t.waitUntil is not a function", source: https://kiwix.app/A/sw.js (9)
2023-12-14 14:30:31.532 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(9)] "Uncaught TypeError: t.waitUntil is not a function", source: https://kiwix.app/A/sw.js (9)
2023-12-14 14:30:31.543 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(17)] "ww init", source: https://kiwix.app/A/sw.js (17)
2023-12-14 14:30:31.545 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(9)] "Uncaught TypeError: t.waitUntil is not a function", source: https://kiwix.app/A/sw.js (9)
2023-12-14 14:30:31.547 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(9)] "Uncaught TypeError: t.waitUntil is not a function", source: https://kiwix.app/A/sw.js (9)
2023-12-14 14:30:31.566 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(42)] "final: https://kiwix.app/A/journals.openedition.org/bibnum/", source: https://kiwix.app/A/load.js (42)
2023-12-14 14:30:31.574 18592-18592 chromium org.kiwix.kiwixmobile I [INFO:CONSOLE(9)] "[object DOMException]", source: https://kiwix.app/A/sw.js (9)
I still don't get what we are speaking about.
Could you please provide
- a URL to the faulty ZIM (because it looks like a ZIM issue)
- the URL (inside the ZIM) where the error occurs (or is it occurring on the home page / main entry of the ZIM?)
As already mentioned by @mgautierfr, what you already found looks a bit like https://github.com/openzim/warc2zim/issues/109
Did you tested the ZIM on another player than Android (i.e. standalone kiwix-serve)? If the issue happens everywhere, I'm afraid you might be loosing everyone time (including yours) on this issue. We are currently developing Zimit2.0 which will get rid of the service worker, so there is a high chance we won't invest any effort in fixing this issue.
@benoit74 This issue is mainly occurring with the Zimit files, mainly the issue with bibnum_fr_all_2021-12.zim and internet-encyclopedia-philosophy_en_all_2023-08.zim
the URL (inside the ZIM) where the error occurs (or is it occurring on the home page / main entry of the ZIM?)
Yes, the issue is with the home page.
Did you tested the ZIM on another player than Android (i.e. standalone kiwix-serve)?
Yes, I have tested it with Android's kiwix-server but there it is showing This page must be loaded via an HTTPS URL to support service worker, and when I tried with https the page URL is not working. But with kiwix-Js extension of the chrome it is working.
We are currently developing Zimit2.0 which will get rid of the service worker, so there is a high chance we won't invest any effort in fixing this issue.
Then this issue will resolved since it might be the issue of Service worker with ZIM file.
But with kiwix-Js extension of the chrome it is working
Please note that no Kiwix JS flavours use the Service Worker provided in the ZIM (because we can't). We provide our own copy, a non-minified version of wabac.js as provided by development warc2zim, which we then minify ourselves for production builds of the PWA versions of these apps (https://pwa.kiwix.org and https://browser-extension.kiwix.org). Moreover, this is run as a Worker rather than a Service Worker, and we call the relevant functions from our own Service Worker.
I think this provides an interesting point of comparison between Kiwix Serve and Kiwix JS. Both are using the same underlying Replay system now, but if it works in Kiwix JS and not in Kiwix Serve, it could indeed point to issues with EITHER the Service Worker OR injection of wombat.js client-side rewriting library.
This issue was mostly related to zimit1. I just created a ZIM of zimit2 and aside few issues linked to zimit2 still being in active development and bibnum needing a custom CSS to hide some stuff not working offline / on desktop (we crawl the ZIM with a mobile profile), the ZIM is fine. bibnum will anyway be included in our test set for final zimit2 validation.
I'm hence going to close this issue which is solved with the move to zimit2