OGA audio file are mirrored in a novid ZIM

[kelson42]

https://en.wikipedia.org/wiki/Laissez-faire

[dcshzj]

Hello! I was exploring this issue and I found that the audio file is not downloaded for the "novid" format. This is the snip of the download log for a normal download of the Laissez-faire article:

Normal download log

[log] [2022-01-11T02:34:44.976Z] Downloading a total of [46] files...
[info] [2022-01-11T02:34:44.977Z] Worker [0] processing batch of [10] files
[info] [2022-01-11T02:34:44.978Z] Downloading [https://en.wikipedia.org/w/extensions/WikimediaBadges/resources/images/badge-validated.png?6232c]
[info] [2022-01-11T02:34:44.978Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/thumb/9/99/Wiktionary-logo-en-v2.svg/40px-Wiktionary-logo-en-v2.svg.png]
[info] [2022-01-11T02:34:44.978Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/thumb/8/8a/Loudspeaker.svg/11px-Loudspeaker.svg.png]
[info] [2022-01-11T02:34:44.979Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/audio-ltr.svg?74d04]
[info] [2022-01-11T02:34:44.979Z] Downloading [https://upload.wikimedia.org/wikipedia/en/thumb/8/8a/OOjs_UI_icon_edit-ltr-progressive.svg/10px-OOjs_UI_icon_edit-ltr-progressive.svg.png]
[info] [2022-01-11T02:34:44.979Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/unwatch-icon-hl.svg?5e652]
[info] [2022-01-11T02:34:44.979Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/unwatch-icon.svg?c5368]
[info] [2022-01-11T02:34:44.979Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/external-link-ltr-icon.svg?48e54]
[info] [2022-01-11T02:34:44.979Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/skins.vector.styles.legacy/images/user-avatar.svg?b7f58]
[info] [2022-01-11T02:34:44.979Z] Downloading [https://en.wikipedia.org/w/extensions/WikimediaBadges/resources/images/badge-problematic.png?f3177]
[info] [2022-01-11T02:34:44.985Z] Worker [1] processing batch of [10] files
[info] [2022-01-11T02:34:44.986Z] Downloading [https://en.wikipedia.org/static/images/mobile/copyright/wikipedia-wordmark-en.svg]
[info] [2022-01-11T02:34:44.986Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/2/23/Icons-mini-file_acrobat.gif]
[info] [2022-01-11T02:34:44.986Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/5/5d/Checker-16x16.png]
[info] [2022-01-11T02:34:44.986Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/portal-separator.png?4ab04]
[info] [2022-01-11T02:34:44.986Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/thumb/f/fa/Wikiquote-logo.svg/34px-Wikiquote-logo.svg.png]
[info] [2022-01-11T02:34:44.986Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/unwatch-temp-icon-hl.svg?f3db6]
[info] [2022-01-11T02:34:44.986Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/magnify-clip-rtl.svg?38fd5]
[info] [2022-01-11T02:34:44.986Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/search.svg?bbf78]
[info] [2022-01-11T02:34:44.987Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/external-ltr.svg?59558]
[info] [2022-01-11T02:34:44.987Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/document-ltr.svg?0bf90]
[info] [2022-01-11T02:34:44.995Z] Worker [2] processing batch of [11] files
[info] [2022-01-11T02:34:44.995Z] Downloading [https://upload.wikimedia.org/wikipedia/en/thumb/0/01/A_coloured_voting_box.svg/16px-A_coloured_voting_box.svg.png]
[info] [2022-01-11T02:34:44.995Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/unwatch-temp-icon.svg?195f5]
[info] [2022-01-11T02:34:44.996Z] Downloading [https://en.wikipedia.org/w/extensions/UniversalLanguageSelector/resources/images/cog-sprite.svg?c3fa1]
[info] [2022-01-11T02:34:44.996Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/thumb/f/f3/Emblem-money.svg/16px-Emblem-money.svg.png]
[info] [2022-01-11T02:34:44.996Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/watch-icon.svg?6ab6f]
[info] [2022-01-11T02:34:44.996Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/bullet-icon.svg?d4515]
[info] [2022-01-11T02:34:44.996Z] Downloading [https://en.wikipedia.org/w/skins/MonoBook/resources/images/bullet.svg]
[info] [2022-01-11T02:34:44.996Z] Downloading [https://en.wikipedia.org/w/extensions/WikimediaBadges/resources/images/badge-silver-star.png?70a8c]
[info] [2022-01-11T02:34:44.996Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/watch-icon-hl.svg?9bf90]
[info] [2022-01-11T02:34:44.996Z] Downloading [https://en.wikipedia.org/w/extensions/WikimediaBadges/resources/images/badge-digitaldocument.png?d1c50]
[info] [2022-01-11T02:34:45.004Z] Worker [3] processing batch of [12] files
[info] [2022-01-11T02:34:45.004Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/thumb/7/76/2006_AEGold_Proof_Obv.png/28px-2006_AEGold_Proof_Obv.png]
[info] [2022-01-11T02:34:45.004Z] Downloading [https://en.wikipedia.org/w/extensions/WikimediaBadges/resources/images/badge-proofread.png?e81f9]
[info] [2022-01-11T02:34:45.004Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/tab-normal-fade.png?1cc52]
[info] [2022-01-11T02:34:45.004Z] Downloading [https://en.wikipedia.org/static/images/project-logos/enwiki-2x.png]
[info] [2022-01-11T02:34:45.004Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/ftp-ltr.svg?c5643]
[info] [2022-01-11T02:34:45.005Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/chat-ltr.svg?e0649]
[info] [2022-01-11T02:34:45.005Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/video.svg?a3758]
[info] [2022-01-11T02:34:45.005Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/thumb/1/1b/Yellow_flag_waving.svg/75px-Yellow_flag_waving.svg.png]
[info] [2022-01-11T02:34:45.005Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/f/f9/En-us-laissez-faire.ogg]
[info] [2022-01-11T02:34:45.005Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/thumb/2/2f/Capitalismlogo.svg/16px-Capitalismlogo.svg.png]
[info] [2022-01-11T02:34:45.013Z] Worker [4] processing batch of [3] files
[info] [2022-01-11T02:34:45.013Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/tab-separator.png?09d4b]
[info] [2022-01-11T02:34:45.014Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/mail.svg?8a79c]
[info] [2022-01-11T02:34:45.014Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/thumb/4/47/Sound-icon.svg/45px-Sound-icon.svg.png]
[info] [2022-01-11T02:34:45.851Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/magnify-clip-ltr.svg?8330e]
[info] [2022-01-11T02:34:46.585Z] Downloading [https://en.wikipedia.org/w/extensions/WikimediaBadges/resources/images/badge-golden-star.png?ed948]
[info] [2022-01-11T02:34:46.587Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/arrow-down.svg?9426f]
[log] [2022-01-11T02:34:47.267Z] Progress downloading files [10/46] [21.7%]
[log] [2022-01-11T02:34:47.271Z] Progress downloading files [20/46] [43.5%]
[log] [2022-01-11T02:34:47.271Z] Progress downloading files [30/46] [65.2%]
[log] [2022-01-11T02:34:47.407Z] Progress downloading files [40/46] [87.0%]
[log] [2022-01-11T02:34:47.411Z] No files to retry
[log] [2022-01-11T02:34:47.411Z] Done with Downloading a total of [46] files

This is the download log with the "novid" format option enabled:

Novid format download log

[log] [2022-01-11T02:35:03.816Z] Downloading a total of [45] files...
[info] [2022-01-11T02:35:03.818Z] Worker [0] processing batch of [10] files
[info] [2022-01-11T02:35:03.818Z] Downloading [https://en.wikipedia.org/w/extensions/WikimediaBadges/resources/images/badge-validated.png?6232c]
[info] [2022-01-11T02:35:03.819Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/thumb/9/99/Wiktionary-logo-en-v2.svg/40px-Wiktionary-logo-en-v2.svg.png]
[info] [2022-01-11T02:35:03.819Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/thumb/8/8a/Loudspeaker.svg/11px-Loudspeaker.svg.png]
[info] [2022-01-11T02:35:03.819Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/audio-ltr.svg?74d04]
[info] [2022-01-11T02:35:03.819Z] Downloading [https://upload.wikimedia.org/wikipedia/en/thumb/8/8a/OOjs_UI_icon_edit-ltr-progressive.svg/10px-OOjs_UI_icon_edit-ltr-progressive.svg.png]
[info] [2022-01-11T02:35:03.819Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/unwatch-icon-hl.svg?5e652]
[info] [2022-01-11T02:35:03.819Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/unwatch-icon.svg?c5368]
[info] [2022-01-11T02:35:03.819Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/external-link-ltr-icon.svg?48e54]
[info] [2022-01-11T02:35:03.819Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/skins.vector.styles.legacy/images/user-avatar.svg?b7f58]
[info] [2022-01-11T02:35:03.819Z] Downloading [https://en.wikipedia.org/w/extensions/WikimediaBadges/resources/images/badge-problematic.png?f3177]
[info] [2022-01-11T02:35:03.829Z] Worker [1] processing batch of [10] files
[info] [2022-01-11T02:35:03.829Z] Downloading [https://en.wikipedia.org/static/images/mobile/copyright/wikipedia-wordmark-en.svg]
[info] [2022-01-11T02:35:03.829Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/2/23/Icons-mini-file_acrobat.gif]
[info] [2022-01-11T02:35:03.829Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/5/5d/Checker-16x16.png]
[info] [2022-01-11T02:35:03.829Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/portal-separator.png?4ab04]
[info] [2022-01-11T02:35:03.830Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/thumb/f/fa/Wikiquote-logo.svg/34px-Wikiquote-logo.svg.png]
[info] [2022-01-11T02:35:03.830Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/unwatch-temp-icon-hl.svg?f3db6]
[info] [2022-01-11T02:35:03.830Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/magnify-clip-rtl.svg?38fd5]
[info] [2022-01-11T02:35:03.830Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/search.svg?bbf78]
[info] [2022-01-11T02:35:03.831Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/external-ltr.svg?59558]
[info] [2022-01-11T02:35:03.831Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/document-ltr.svg?0bf90]
[info] [2022-01-11T02:35:03.842Z] Worker [2] processing batch of [11] files
[info] [2022-01-11T02:35:03.842Z] Downloading [https://upload.wikimedia.org/wikipedia/en/thumb/0/01/A_coloured_voting_box.svg/16px-A_coloured_voting_box.svg.png]
[info] [2022-01-11T02:35:03.842Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/unwatch-temp-icon.svg?195f5]
[info] [2022-01-11T02:35:03.842Z] Downloading [https://en.wikipedia.org/w/extensions/UniversalLanguageSelector/resources/images/cog-sprite.svg?c3fa1]
[info] [2022-01-11T02:35:03.842Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/thumb/f/f3/Emblem-money.svg/16px-Emblem-money.svg.png]
[info] [2022-01-11T02:35:03.843Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/watch-icon.svg?6ab6f]
[info] [2022-01-11T02:35:03.843Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/bullet-icon.svg?d4515]
[info] [2022-01-11T02:35:03.843Z] Downloading [https://en.wikipedia.org/w/skins/MonoBook/resources/images/bullet.svg]
[info] [2022-01-11T02:35:03.843Z] Downloading [https://en.wikipedia.org/w/extensions/WikimediaBadges/resources/images/badge-silver-star.png?70a8c]
[info] [2022-01-11T02:35:03.843Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/watch-icon-hl.svg?9bf90]
[info] [2022-01-11T02:35:03.843Z] Downloading [https://en.wikipedia.org/w/extensions/WikimediaBadges/resources/images/badge-digitaldocument.png?d1c50]
[info] [2022-01-11T02:35:03.852Z] Worker [3] processing batch of [11] files
[info] [2022-01-11T02:35:03.853Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/thumb/7/76/2006_AEGold_Proof_Obv.png/28px-2006_AEGold_Proof_Obv.png]
[info] [2022-01-11T02:35:03.853Z] Downloading [https://en.wikipedia.org/w/extensions/WikimediaBadges/resources/images/badge-proofread.png?e81f9]
[info] [2022-01-11T02:35:03.853Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/tab-normal-fade.png?1cc52]
[info] [2022-01-11T02:35:03.853Z] Downloading [https://en.wikipedia.org/static/images/project-logos/enwiki-2x.png]
[info] [2022-01-11T02:35:03.853Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/ftp-ltr.svg?c5643]
[info] [2022-01-11T02:35:03.853Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/chat-ltr.svg?e0649]
[info] [2022-01-11T02:35:03.853Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/video.svg?a3758]
[info] [2022-01-11T02:35:03.854Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/thumb/1/1b/Yellow_flag_waving.svg/75px-Yellow_flag_waving.svg.png]
[info] [2022-01-11T02:35:03.854Z] Downloading [https://en.wikipedia.org/w/extensions/WikimediaBadges/resources/images/badge-golden-star.png?ed948]
[info] [2022-01-11T02:35:03.854Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/arrow-down.svg?9426f]
[info] [2022-01-11T02:35:03.860Z] Worker [4] processing batch of [3] files
[info] [2022-01-11T02:35:03.860Z] Downloading [https://en.wikipedia.org/w/skins/Vector/resources/common/images/tab-separator.png?09d4b]
[info] [2022-01-11T02:35:03.860Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/mail.svg?8a79c]
[info] [2022-01-11T02:35:03.860Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/thumb/4/47/Sound-icon.svg/45px-Sound-icon.svg.png]
[info] [2022-01-11T02:35:04.767Z] Downloading [https://en.wikipedia.org/w/resources/src/mediawiki.skinning/images/magnify-clip-ltr.svg?8330e]
[info] [2022-01-11T02:35:05.017Z] Downloading [https://upload.wikimedia.org/wikipedia/commons/thumb/2/2f/Capitalismlogo.svg/16px-Capitalismlogo.svg.png]
[log] [2022-01-11T02:35:05.977Z] Progress downloading files [10/45] [22.2%]
[log] [2022-01-11T02:35:05.978Z] Progress downloading files [20/45] [44.4%]
[log] [2022-01-11T02:35:05.992Z] Progress downloading files [30/45] [66.7%]
[log] [2022-01-11T02:35:06.267Z] Progress downloading files [40/45] [88.9%]
[log] [2022-01-11T02:35:06.268Z] No files to retry
[log] [2022-01-11T02:35:06.269Z] Done with Downloading a total of [45] files

There is one less file downloaded (En-us-laissez-faire.ogg), which is correct as it is the spoken version of the article at the bottom. However, I also noticed that in both cases, the "listen" link at the top of the article is clickable, but nothing happens when a user clicks on it.

Do you have any pointers on how to proceed with this issue? I would love to help out as a new contributor. Thank you!

[kelson42]

Hmm.... it is there available http://library.kiwix.org/wikipedia_en_all_maxi/A/Laissez-faire (ZIM file is https://mirror.download.kiwix.org/zim/wikipedia/wikipedia_en_all_maxi_2021-12.zim) and see the tags http://library.kiwix.org/wikipedia_en_all_maxi/M/Tags. This is done with novid as well. Here is the recipe of the Zimfarm https://farm.openzim.org/recipes/wikipedia_en_all_maxi. If novid is specified no audio/video file should be download at all. A clickable link pointing to HTTP 404 is wrong as well, maybe both problems are linked... or not.

[ghost]

Hello, I am new here and want to contribute in your project I would like to ask what i should learn to contribute here and please also tell me some resources ....

[Aadish01]

Hello , is this issue still open or assigned to someone ?

[kelson42]

yes, this is open

[Aadish01]

Can you please tell which file needs to be fixed ??

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[kelson42]

It seems to me this might be a duplicate of #890

[kelson42]

So here we have in the HTML scraped from Wikipedia something like:

<a href="//upload.wikimedia.org/wikipedia/commons/7/7e/Fr-Laissez-faire.oga" class="internal" title="Fr-Laissez-faire.oga">listen</a>

If include (so no special --format, then the link should point to the data in the ZIM

If not-included, then the link should be removed, but content "listen" should be kept (like for a read/dead internal link).

listen

Obviously we should ideally do something smarter than that (don't display "listen" with the logo before), but this is an other topic.

[kelson42]

Fixed by #1763