zim-requests icon indicating copy to clipboard operation
zim-requests copied to clipboard

New request: The Terraria Wiki

Open PoggChamp opened this issue 2 years ago • 27 comments

  • Website URL: https://terraria.wiki.gg/
  • License: Creative Commons
  • Desired ZIM Title: Terraria Wiki
  • Desired ZIM Description: the comprehensive reference written and maintained by the players.
  • Desired ZIM Icon –png (URL or attach one): https://terraria.wiki.gg/favicon.ico
  • Language (ISO 639-3): eng
  • Is this a MediaWiki?: yes

PoggChamp avatar Oct 29 '23 17:10 PoggChamp

The recipe created https://farm.openzim.org/recipes/terrariawiki_en Will update the library link once ready

RavanJAltaie avatar Nov 15 '23 14:11 RavanJAltaie

Troubleshooting the recipe https://github.com/openzim/mwoffliner/issues/1948

RavanJAltaie avatar Nov 23 '23 14:11 RavanJAltaie

I'd like to see a Terraria wiki zim file since Terraria had a final update. Would be useful to put on my iiab.

outtieTV avatar Sep 01 '25 23:09 outtieTV

@outtieTV Site seems to be down/moving, and from the mwoffliner issue above it looks like it has a Cloudflare in place that will prevent us from crawling. Any chance you can reach out to them (cc [email protected]) and ask them to whitelist us?

Popolechien avatar Sep 02 '25 06:09 Popolechien

Last attempt on Zimfarm was a bit old, I've just requested it again to give it a fresh try

benoit74 avatar Sep 02 '25 08:09 benoit74

I've just requested a new task with latest mwoffliner, we've again made a significant amount of changes : https://farm.openzim.org/pipeline/34c933fd-ddb5-4efc-a134-5145f0893d92

And a bug was reported at Broken page layout in Terraria wiki (wiki.gg) ZIM at large widths (>900px, >=1120px) mwoffliner#2562

benoit74 avatar Oct 17 '25 14:10 benoit74

I'll have a proper long stroll around the new file (2025-10) in the coming days, but I've had a quick look just now. The design suffered a bit (its nice background art is always obscured now), but the content I've quickly looked at is now displayed reliably. Some spacing from screen edges might be nice, but that's very much a cosmetic thing.

Thanks @benoit74!

D-side avatar Oct 17 '25 17:10 D-side

Spotted a problem in the latest file, math doesn't render, such as formulas on page regarding Terraria's luck mechanic.

The TeX markup is still there as well as [math] and [/math] delimiters, but they appear in the content as plain unrendered source text in muted colors.

I'm a pretty casual player so I don't find this very important, but especially dedicated players might be expecting to see rendered formulas.

D-side avatar Oct 18 '25 19:10 D-side

Indeed, see https://dev.library.kiwix.org/viewer#terrariawiki_en_maxi_2025-10/Luck

@benoit74 Are we impacted here by https://github.com/openzim/mwoffliner/issues/1371 ?

kelson42 avatar Oct 19 '25 07:10 kelson42

Another difference I've noticed is a lack of sounds, e. g. see the info card at the top for Meowmere in the original and in the new file, the section titled "Sounds".

edit: music, with a different kind of player, is missing too: original, zim.

Also noticed that table of contents is missing (in the music page at least), though heading anchors seem to work.

D-side avatar Oct 19 '25 11:10 D-side

Regarding math formula, we are indeed impacted by https://github.com/openzim/mwoffliner/issues/1371 ; hopefully will be fixed in somewhere in 2026.

Regarding absence of sound, it is normal, the Zimfarm recipe requested a novid:maxi flavor, and novid means no video and no sound. Is sound very important or a nice to have? It will probably significantly increase the ZIM size. Should we have two ZIMs, one with sound and one without?

Regarding the table of content, this is normal as well, table of contents are to be handled by the ZIM reader, and not all ZIM readers have implemented it equally well.

benoit74 avatar Oct 20 '25 05:10 benoit74

Ohh, right, for a novid flavor that makes sense.

I'm not sure the file size would be that much bigger with sound included, the game's total size is under 700MB and I don't think it employs any asset generation, though different formats I suppose could cause an increase beyond that, in theory. The game also reuses many of its SFX for many purposes and from what I can tell, the reuse is reflected on the wiki (pages using the same effect point at the same sound file). Most of the sounds (almost all except the music) are also very short. Sound cues in the game are very optional, but helpful, so those would be nice to have.

And the current ZIM file already has a number of very video-like GIF animations such as 3.45Mb demonstration of the Scarab Bomb (a lower floating block on the right). I'd guesstimate that those would still be the top contributor to the file size even with sounds. In fact, there might be substantial compression potential in reencoding those from GIF to video. I tried a plain ffmpeg -i $in.gif $out.mp4 without any specific options and it squished those 3.45MB to just 1.05MB (using H264) and I'm not noticing any loss in quality.

I'd try building a ZIM file with sounds for comparison. Maybe there's not much point in the novid flavor in the first place.

D-side avatar Oct 20 '25 07:10 D-side

I've just requested a ZIM with sounds and videos: https://farm.openzim.org/pipeline/547bb059-bcba-4b44-974c-4d1d8e2f4564

We propose with @Popolechien to publish the ZIM despite Mathjax issue, it is not a blocker point from our PoV.

Let's see if ZIM with sound has an appropriate size, if yes we will publish this one, otherwise I would suggest to create both novid and all ("with vid") ZIMs..

benoit74 avatar Oct 30 '25 13:10 benoit74

The size difference ended up being 1.26GiB maxi (I assume it's novid) vs. 1.40GiB for the whole thing, which is probably negligible. And sounds work fine.

I also don't see MathJAX as critical for that particular wiki. I take it it's a bigger issue in other wikis so there are hopes of fixing this eventually and that the fix would eventually reach Terraria wiki as well. That sounds fine by me.

An additional bit of context on future updates: since the game is being updated to this day (with a new 1.4.5 update currently in the works), periodic updates would be helpful. Not until the next update probably, but certainly for some time after it releases. If feasible, of course.

D-side avatar Oct 30 '25 20:10 D-side

@D-side thank you

We aim to update the ZIM quarterly. It is fully automated. It allows to reasonably grab any edits made in the Wiki, and to benefit from any new feature / bug fix done in the scraper.

You can always request (with a new issue in this Github Repo) a manual run between these quarterly updates if for some reason the ZIM is too outdated because many edits have been done in the wiki (typically following a major release). But we prefer when we can avoid them to save our efforts for other stuff ^^

benoit74 avatar Oct 31 '25 06:10 benoit74

@benoit74 understood, thanks! All set for publishing the full ZIM file then?


Also, regarding the idea I mentioned above about converting *.gifs to *.mp4s for some space savings, I take it mwoffliner would be the right place for it?

D-side avatar Nov 01 '25 06:11 D-side

Yes, mwoffliner is the place to implement it. You can open an issue there to discuss the matter first, not sure it is really that straightforward

benoit74 avatar Nov 01 '25 13:11 benoit74

on stats with master/expert e.g. monsters pages clicking the button (master/expert) doesnt update the content

GitEin11 avatar Nov 02 '25 14:11 GitEin11

Do you have a URL to share, or the precise title of the article at least?

I strongly suspect this is a Javascript "thing", and Javascript is not supported ATM (see https://github.com/openzim/mwoffliner/pull/2483)

benoit74 avatar Nov 02 '25 17:11 benoit74

Do you have a URL to share, or the precise title of the article at least?

I strongly suspect this is a Javascript "thing", and Javascript is not supported ATM (see openzim/mwoffliner#2483)

any monster page, well I guess we'll gonna wait till JavaScript is supported.

https://github.com/user-attachments/assets/aa0d9131-9601-4ed4-9486-4b2b87c8dc1f

GitEin11 avatar Nov 02 '25 18:11 GitEin11

Yep.

Example: original, latest file (click the colored Classic/Expert/Master inside the info card on the right)

D-side avatar Nov 02 '25 19:11 D-side

Yep, this is JS code... not sure yet where it comes from, but you will have to live without it indeed.

benoit74 avatar Nov 02 '25 20:11 benoit74

It's probably a bigger issue compared to sound or math, as "Expert mode" has a number of exclusive things, which makes some of the tabs' contents very different, particularly monster item drops. And someone who might want an offline version of Terraria Wiki would probably be interested in the game enough to play on Expert or Master.

So I'd suggest postponing proper publication until this is sorted out.

D-side avatar Nov 02 '25 22:11 D-side

@D-side @GitEin11 Do you reckon we nevertheless have enough to move this to PROD, or would you rather wait until the above issues are fixed (meaning that for the time being and without these the ZIM is nice but not super useful)?

Popolechien avatar Nov 03 '25 07:11 Popolechien

I'm on the fence about this, leaning towards "wait".

IMO it is useful for many purposes in its current form, but missing information on higher difficulties (as a result of missing JS) is a significant omission. I'm not sure what kinds of breakages are expected from the main library, but in this state I wouldn't announce its existence in Terraria communities (hypothetically, I'm not actively participating in any right now, this is solely to indicate my level of doubt). Is this enough of a bar to wait?

D-side avatar Nov 03 '25 08:11 D-side

I think it is still good to make the current version available, their still a work around to access exclusive items thru the article named "expert-exclusive content", although the stats like monster hp and such will still be missing

GitEin11 avatar Nov 03 '25 13:11 GitEin11

Ok let's wait a bit then.

Popolechien avatar Nov 07 '25 10:11 Popolechien