kiwix-serve should be localised

[kelson42]

[mgautierfr]

Yes, it would be nice. gettext is the tool for that.

However, it may break one of our "functionality" : The "one" file binary (static build) Translation will be store at separated files in the fs. We could embedded the translation files as embedded resources in the binary, but I don't know how gettext would access them (hoping it is possible)

[kelson42]

@mgautierfr We need a solution where the translations are in the binary itself.

[kelson42]

For the string format, we should do like for kiwix-desktop

[kelson42]

@veloman-yunkan Should work like for kiwix-desktop

[veloman-yunkan]

I guess this is going to be a little more complex than kiwix-desktop.

The text displayed in the front-end has 4 origins:

1. mustache templates at libkiwix/static/templates/*.html
2. libkiwix/static/skin/index.js
3. C++ source code 3.1 libkiwix/src/server/internalServer.cpp: explanations of some errors, as well as the suggestion system suggesting to run a full text search by adding the word "containing" to the query. 3.2 texts of exceptions thrown from libkiwix/src/book.cpp and libzim (!!!)
4. content of ZIM files

Of course, here we are concerned only with items 1, 2, and 3. Note that I have assumed that the OPDS output should NOT be localized (and thus omitted libkiwix/static/templates/*.xml from the above list).

Regarding the mustache templates - I think that we should translate each template in its entirety rather than extract text from it and maintain it as separate translatable resource. The reason is simple - different languages have different grammar (and therefore structure of sentences). But this will complicate the translators' job (who will have to deal with mustache and HTML syntax) as well as be a significant maintenance burden (when the main template is modified, the translated templates will have to be updated correspondingly).

Regarding libkiwix/static/skin/index.js - that one should be easy unless we want to respect its ambition of being smart and producing grammatically correct English when reporting the count of books ("1 book" vs "2 or more books"). There are languages with multiple plural forms (something like 1 book, 2 booka, 3 booky, 4 or more books). Should we target that during our localization effort? A general purpose internationalization solution for javascript apps is something like http://i18njs.com/, but for our current index.js it may be an overkill.

And finally should we go after the few errors coming from the C++ code (including libzim) and the "containing 'search terms'" phrase in suggestions?

[kelson42]

@veloman-yunkan The Kiwix team does not translate things, this is the duty of the Translatewiki team. As such we have to adapt to the Translatewiki workflow and this is conceived to translate "simple" UI strings. As such I believe we should include strings in template and not translate templates. If you think doing so for a concrete template might lead to problems, let me know about this template.

Regarding plural problems, I'm not an expert, but this should be possible to handle this properly. See https://translatewiki.net/wiki/Plural. We already had a full discussion about that on a Kiwix-Android ticket.

The overall approach is that everything which is end-user visible (so not the errors on the command line) should be translated. All the rest can be kept in English.

Including @Nikerabbit, he might be of good advise.

[veloman-yunkan]

@veloman-yunkan The Kiwix team does not translate things, this is the duty of the Translatewiki team. As such we have to adapt to the Translatewiki workflow and this is conceived to translate "simple" UI strings. As such I believe we should include strings in template and not translate templates. If you think doing so for a concrete template might lead to problems, let me know about this template.

I didn't know anything about translatewiki.net except the it is used to internationalize kiwix-desktop. My simplistic assumption was that it only supports translating plain text messages. Now I see that they support messages with {{parameters}} and HTML markup (of course nothing prevented us from having those in our messages anyway, but the good thing is that translators should not be confused by such syntax since it is a supported convention). So nothing prevents us from extracting fragments of our mustache templates as translatable messages.

Regarding plural problems, I'm not an expert, but this should be possible to handle this properly. See https://translatewiki.net/wiki/Plural. We already had a full discussion about that on a Kiwix-Android ticket.

I read the mentioned discussion about usage of plurals in messages. Translatewiki lists Kiwix as not-supporting plurals though that info may be outdated.

• As of March 2012, plural formatting is not needed on any Kiwix messages, and is not yet supported on Kiwix.

I wonder if we should go after setting up plurals for kiwix-serve now or address it later (if at all).

[veloman-yunkan]

Need to make sure that localization of kiwix-serve plays well with caching.

[kelson42]

@veloman-yunkan Do I’m right to say that all the preparatory work needed to implement this ticket has meanwhile been done?

[veloman-yunkan]

@kelson42 Not exactly. Introduction of the iframe-based viewer shifted some burden of localization from backend to frontend.

[kelson42]

@veloman-yunkan Do we have blockers in other tickets? which one?

[veloman-yunkan]

@kelson42 I will start working on the localization of the front-end.