operations

operations copied to clipboard

Currently there are regular dumps of the wiki in xml format to https://wiki.openstreetmap.org/dump/. These are useful mainly for statistical analysis and similar uses. To be able to provide the content in a more useful form for offline use we should be producing dumps in ZIM format that then could be read with Kiwix @Kiwix

The tool of choice to generate a ZIM file would currently seem to be https://github.com/openzim/mwoffliner

We talked about a zim version in #536. I didn't have much luck getting the zim exporter up and running, though (due to some issues with the import step, which was a prerequisite for a zim export).

It is naturally by far not a new issue, I first suggested it years ago (iirc that is the reason we have the current dumps). The alternative to mwoffliner would seem to be https://github.com/openzim/zimit Likely the best thing would be to ask the folks at @Kiwix for input.

I think this might work when you create a zim export based on the productive osm wiki database.

As a downstream consumer with only a Wiki dump available, even importing this dump into your own Mediawiki instance takes forever, and is totally non-feasible as of now. Needless to say that isn't exactly a good starting point to think about a zim export.

Progress has been slow. But I think I am nearly there. Resolved many bugs with wiki, primarily timeout (on huge pages) and some templates causing 500 errors.

I've had a successful run of: openstreetmap-wiki_en_all_nopic_2022-08.zim I am still trying to get maxi to successfully complete.

There is a successful run here: https://mirror.download.kiwix.org/zim/other/openstreetmap-wiki_en_all_nopic_2022-11.zim The maxi version (with pictures) is still failing.

Now documented: https://wiki.openstreetmap.org/wiki/Using_OpenStreetMap_offline#Offline_File

Thanks @Firefishy ... just one thing, I'm not quite sure what There is currently a slight problem with the creation of the files in that it sometimes doesn't transfer correctly, after having not been able to build for multiple months is supposed to imply, maybe some old text not edited out?

Thanks @Firefishy ... just one thing, I'm not quite sure what There is currently a slight problem with the creation of the files in that it sometimes doesn't transfer correctly, after having not been able to build for multiple months is supposed to imply, maybe some old text not edited out?

Text is not by me.

This text was added by @kmpoppe

That text is literally a reminder to not taking for granted that the process runs without hick ups right now.

I also started my own nopic run on my server to check whether the run failing in the ZIMfarm is due to their docker setup or anything else.

I will keep this is under surveillance over the next weeks and then edit the text accordingly.

Could any of the watchers hint to whether the zim files could be mirrored in the OSM infrastructure in case kiwix fails to keep them?

Kai