i18n/l10n roadmap

[siddharthvp]

Across modules:

• Abstract the code that extracts the filled-in form data into a class or function (defined in Morebits). In some modules (like xfd), we may want to allows per-wiki configurations that add new input fields. (already done in #908)
• Use HTML5 required field attribute for validating required fields in most places. This does away the need for the alerts, which would be one less thing to localise.
• Document Twinkle's dependencies on wiki templates
• Extend use of Morebits.taskManager, or another promise-based system, across most modules.
• Use object-oriented pattern in the more complex modules like Tag, XFD and CSD to make them extensible more easily.
• Set up a build step for transpiling ES2020 code. This makes working with classes a lot easier.
• Explore use of unit testing in module code, probably based on SinonJS and/or web driver tools, which would significantly reduce the time and effort needed for testing. Integrate unit tests in Travis.
• Explore the use of translatewiki.net for translation of strings.
• Set up a system for automated reporting of unexpected errors (such as TypeErrors, unexpected API errors, etc). This would help us identify causes of issues without having to get users to provide stack traces.

Module components

What do Twinkle modules have in common? They are all tools with a GUI containing a form in which the user can enter some data. On submitting, a series of actions are executed using the API. (Fluff is an exception -- it's completely different. But fluff is not project specific, and can be readily i18n-ed. So that is the least of our concerns.)

• UI
  • Morebits.quickForm, Morebits.simpleWindow are used for creating the UI. No changes are planned here.
• Collecting form data
  • With #908, we have a structured way of extracting form data in one line of code. No further changes planned here.
• Actions
  • Morebits.wiki.api and Morebits.wiki.page are used for the bulk of actions code. To overcome the issues mentioned in #1029 and #911, I plan to rewrite these two in modern JS with unit tests and documentation. I believe having a rich and well-documented library of reusable components is one of the best ways to enable l10n of code involving processes that are wildly different across wikis.
  • Orchestrating the actions in a proper sequence, handling failures, etc is something Twinkle does the worst, leading to bugs like #726, this and this (also present for speedy and other modules). I think the solution is a promise-based system like Morebits.taskManager (already implemented PR #1036, but not merged yet), which needs to be used throughout.
• Showing status messages
  • Uses Morebits.status, integrated into Morebits.wiki.page and Morebits.wiki.api
  • A lot of bugs are present (like #1049) but should be easy to fix.

Per the MVC design pattern, the UI can be considered the view, actions can be considered the model, collecting form data and displaying status messages may be considered the controller.

Specific files and modules

• Twinkle.js: Base file that adds the Twinkle menu to the DOM and reads user preferences. Would be shared across wikis.
• Morebits.js: GUI and API library. The API library would be amended so that it uses errorformat=html, which does away the need for localising error messages (it would show MW namespace messages from the local wiki). The remaining few strings would need i18n.
• Fluff, diff, unlink, batchdelete, batchundelete, batchprotect: (Easy)
  • These modules don't have any data or project-specific logic so i18n would be just allowing translation of strings.
• Tag: Choice of tags shown should be configurable by namespace, whether the page is a redirect, and other factors. Some may require additional inputs from user. Some auxiliary actions may be associated with some tags ({{merge}}, etc).
• SharedIP: Requires additional inputs from user for each tag. Merge the data in Twinkle.shared.callback.change_shared into Twinkle.shared.standardList.
• XFD: This is essentially a collection of 6 different modules, one for each XFD venue. Each follows different conventions. Not very suitable for l10n.
  • Since every deletion process is different, I think it would be silly to have one software that (tries to) handles them all. The best we can do is to abstract out all common functionality in classes/functions (eg. notifying a user, adding a tag to a page, logging in a userspace log) and let individual wikis reuse them to write their own code.
  • Provide classes providing base functionality which different wikis and XFD venues can extend.
• PROD, Deprod
  • These are enwiki-specific. Equivalent of a PROD process doesn't exist on most wikis, so out of scope. Wikis which do have such a process will be able to provide this functionality by extending the classes for XFD processes.
• CSD: Choice of deletion templates to be shown should be configurable per namespace, whether the page is a redirect and other factors. Existing code is quite messy (for instance, there are multiple instances of variables initialised like arrays but later used like objects!). Might need to start from scratch - create a lightweight speedy module with no radioClick mode and no sysop mode. Then, add a sysop mode into it. Data on namespace modes, the criteria and templates available for each mode used will be made localisable.
• DI (image deletion): This is an extension of CSD module. It's unlikely to be needed on other wikis in addition to a localised CSD module.
• ARV: Investigate ARV processes on other wikis, assess whether it's possible or necessary to localise this. This will need to be approached the same way as XFD if the processes on different wikis are quite different.
• Warn, Welcome, Talkback: These modules basically add a substituted template to a user talk page.
  • Welcome: Put all template data into a single object (already done in #747)
  • Talkback: Put all configurable data into a single object (#758 achieves this partially)
  • Warn: A tough one considering that most wikis won't have a 4-stage warning process like on enwiki, which appears to be quite integrated in the code.
• Config: A very well-structured module. L10n would be fairly easy.
• Block: The data on templates used and presets will be made localisable. The toggle_see_alsos hook will be made customisable.
• Protect: Not reviewed fully, but the data on tagging templates and presets needs l10n.

Translation of strings

• As a first step, separate all the message strings from the code and project-specific data and put them in an object, in the same file. Create a function that reads the messages from the the messages object, and injects any variables (named $1, $2, etc) into them. This first step will make it easy to move the everything to a separate file later on.
• Next, actually separate i18n/l10n data for each module to an associated file. Since these won't be just strings, they'll have to be JS files rather than JSON. While we could put all messages across modules in a separate JSON file, I don't think that would be workable since the i18n can depend on the l10n.

[Huji]

Related to the i18n of Morebits: It also contains a great deal of English- or LTR-specific features (messages, even the "question mark" for the tooltips being in LTR ? and not customizable to an RTL ؟ and so on).

[siddharthvp]

Only the status messages in Morebits.wiki.page and Morebits.wiki.api in morebits are language-specific I believe. Those can be handled by letting wikis override the English defaults. The question mark in tooltips is hard-coded here -- I suppose that can also be handled likewise.

Using errorformat=html in the API requests, the error messages shown on hard error cases can be switched to the HTML versions from the MediaWiki namespace of the wiki -- which should already be localised.

[Huji]

The relatively recent transition from a PNG-based question mark to an actual question mark character was definitely a step forward in Morebits. If we could make that a variable that can be overridden easily, that'd be nice.

[Amorymeltzer]

I'm I right in understanding that your goal is for the grant to tackle/supersede #911, #1029, and #1036 as part of this?

[siddharthvp]

@Amorymeltzer Pretty much yeah. #911 will be definitely superseded by the "new morebits". I will try to extend #1036 to cover all major modules, most importantly XFD which needs it badly.

[MusikAnimal]

It sounds closer to a complete rewrite, but well thought out! If you are confident you can take all of this on within 3 months time, I see no reason to stop you, so long as others here are also in support. I probably won't be able to help but so much with code review, etc., I'm afraid, so bear that in mind.

Might I suggest we document the code with JSDoc, preferably adding a build step to require every method be documented? Later we can generate and publish the docs somewhere. I also recommend babel-plugin-add-header-comment to leave file comment blocks post-transpilation, since depending on what you do Babel might put some ugly polyfill code at the very top of every file.

The only other thing I worry about is non-critical development on master will basically have to halt for these 3 months, right? It may be difficult to keep the branches in sync. Unless you think we can do this piecemeal and continually merge into master? There's also a lot of unmerged code as it is. What will happen to that?

[siddharthvp]

Excited to announce that the grant was approved! Beginning work in a day or two, beginning with the Tag and XFD modules.

[siddharthvp]

Notes on adding a string i18n library is at https://github.com/wikimedia-gadgets/twinkle-core/issues/1.

Also belatedly noting here that the work has shifted to https://github.com/wikimedia-gadgets/twinkle-core and https://github.com/wikimedia-gadgets/twinkle-enwiki, the former being the shared core and latter the enwiki localisation prototype.

[siddharthvp]

I filed a request for setting up twinkle on translatewiki.net a few days back - https://phabricator.wikimedia.org/T278627. If anyone wishes to help with writing message documentations in the qqq.json file, I would appreciate.

[MusikAnimal]

Just make sure all translations are reviewed before deployment. Using markup in the messages like <strong> suggests these messages are not escaped and are vulnerable to XSS. Alternatively instead of doing i.e. Latest revision <strong>$1</strong> you could do just do Latest revision $1 and wrap the $1 parameter with <strong> in the code, as with msg('key', `<strong>${param}</strong>`)

[siddharthvp]

HTML is actually only supported for messages that go into Morebits.status or labels of Morebits.quickForm elements. They both use $.parseHTML – which scrubs out any

[Huji]

Hi all. Can you please provide some guidance as to how to use jquery.i18n on Wikipedia gadgets? I tried dependencies=jquery.i18n in gadget definition but that did not help ($.i18n was still undefined).

We are trying to create a local translation for Morebits, now that it supports i18n, and we are trying to use jquery.i18n for it. If there is a step-by-step instruction for how to translate Morebits and/or an actual use case on any wiki, please point me to that as well.

[siddharthvp]

$.i18n is definitely defined after mw.loader.using('jquery.i18n'), so adding it to dependencies should be enough.

However, jquery.i18n is an unmaintained library. It's superseded by https://github.com/wikimedia/banana-i18n – I recommend using that instead as it's modern, modular and doesn't depend on globals for maintaining state. But it's not available in MW itself, so you'd need to transpile it to ES5 and host it locally as a hidden gadget. There's also https://github.com/wikimedia-gadgets/orange-i18n, a leaner fork of the same that does some optimisations for MW environment.

[Huji]

Can you set up orange-i18n on test.wikipedia.org and use it in a small gadget (could be a simple gadget with only one translatable message) so I can see the end-to-end process in action? I am a bit confused as to how exactly to get it to work.

I am an interface admin on that wiki so I can assist with copying code to the MediaWiki namespace if needed.

On Mon, Dec 27, 2021 at 10:53 AM Siddharth VP @.***> wrote:

$.i18n is definitely defined after mw.loader.using('jquery.i18n'), so adding it to dependencies should be enough.

However, jquery.i18n is an unmaintained library. It's superseded by https://github.com/wikimedia/banana-i18n – I recommend using that instead as it's modern, modular and doesn't depend on globals for maintaining state. But it's not available in MW itself, so you'd need to transpile it to ES5 and host it locally as a hidden gadget. There's also https://github.com/wikimedia-gadgets/orange-i18n, a leaner fork of the same that does some optimisations for MW environment.

— Reply to this email directly, view it on GitHub https://github.com/wikimedia-gadgets/twinkle/issues/1144#issuecomment-1001625875, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAUULTAME64FFCXEMDM64LUTCCLZANCNFSM4SKDEIEA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

[Huji]

PS: I made https://test.wikipedia.org/wiki/MediaWiki:Gadget-Morebits-i18n.js but it displays the fallback message (not the localized message). What am I doing wrong? And how can I transition it from jquery.i18n to banana or orange?

[siddharthvp]

@Huji it seems to work for me? See https://ibb.co/F4ZkGG7 I see localised-string in the console.

As for setting up orange-i18n as a gadget, yes I can help with this - give me some time. It isn't very straightforward if I recall correctly. My intadmin access on testwiki has expired yet again. I see you're also a bureaucrat, can you help with that?

[Huji]

Yes. Can you please leave a message for me on test.wikipedia.org so I can use it as the basis of giving you the right?