add ability to translate the user guide on Crowdin

[michaelDCurran]

Link to issue number:

None.

Summary of the issue:

Currently user documentation such as the user guide are made translatable via a custom (and very old) translation system hosted by NV Access. For many reasons we need to move away from this old system to something more mainstream and maintainable. We have already successfully moved translation of NVDA interface messages to Crowdin, and we should do the same for the user guide and other documentation.

Description of user facing changes

None.

Description of development approach

• Added markdownTranslate.py, which contains several commands for generating and updating xliff files from markdown files. These xliff files can then be uploaded to Crowdin for translation, and eventually downloaded again and converted back to markdown files. Commands include:
  • generateXliff: to generate an xliff file from a markdown file. Firstly a 'skeleton' of the markdown file is produced which is all the structure of a markdown file, but the translatable content on each line has been replaced by a special translation ID. Lines such as blank lines, hidden header rows, or table header separator lines are included in the skeleton in tact and are not available for translation. The xliff file is then produced, which contains one translatable string per translation unit, keyed by its respective translation ID. Each unit also contains translator notes to aide in translation, such as the line number, and any prefix or suffix markdown structure. E.g. a heading might have a prefix of ### and a suffix of {#Intro}. The skeleton is also embedded into the xliff file so that it is possible to update the xliff file keeping existing translation IDs, and or generate the existing markdown file from the xliff file.
  • generateMarkdown: Given an xliff file, a the original markdown file is reproduced from the embedded skeleton, and either the translated or source strings from the xliff file, depending on whether you want a translated or untranslated markdown file.
  • updateXliff: to update an existing xliff file with changes from a markdown file, ensuring that IDs of existing translatable strings are kept in tact. This command extracts the skeleton from the xliff file, makes a diff of the old and new markdown files, then applies this diff to the skeleton file I.e. removes skeleton lines that were removed from the markdown file, and adds skeleton lines (with new IDs) for lines that are newly added to the markdown file. All existing lines stay as is keeping their existing translation IDs. Finally a new xliff file is generated from the up to date markdown file and skeleton, resulting in an xliff file that contains all translatable strings from the new markdown file, but reusing translation IDs for existing strings.
  • translateXliff: given an xliff file, and a pretranslated markdown file that matches the skeleton, a new xliff file is produced containing translations for all strings.
  • pretranslateAllPossibleLangs: this walks the NVDA user_docs directory, and for each language, pretranslates the English xliff file using the existing pretranslated markdown file from the old translation system (if it matches the skeleton exactly) producing a translated xliff file that can be uploaded to Crowdin to bring an existing translation up to where it was in the old system.
  • runTests: runs some tests using 2024.2 and 2024.3 English and French user guides to ensure files are translated and generated as expected.
• Added a generated xliff file for the current English user guide markdown file. Note that this has been uploaded to Crowdin for translation.
• Added a GitHub action that runs on the beta branch if English userGuide.md changes. The action regenerates the original markdown file from the current English user guide xliff, then updates the xliff file based on the changes from the original markdown file to the current markdown file. This xliff file is then uploaded to Crowdin, and also committed and pushed to beta.

Testing strategy:

• Uploaded the English user guide xliff to Crowdin.
• pretranslated a French xliff file from the English user guide xliff and a pretranslated French markdown file.
• Uploaded the pretranslated French xliff file to Crowdin for French.
• Downloaded the latest French xliff file from Crwdin, ensuring all strings were translated.
• Added / removed several lines from the English user guide markdown file and committed changes.
• Ensured the GitHub action ran with no errors.
• Downloded the French xliff from Crowdin and confirmed that the new added strings were available for translation, the removed strings were no longer available, and that all existing strings were still translated appropriately.

Known issues with pull request:

None known.

Code Review Checklist:

• [x] Documentation:
  • Change log entry
  • User Documentation
  • Developer / Technical Documentation
  • Context sensitive help for GUI changes
• [x] Testing:
  • Unit tests
  • System (end to end) tests
  • Manual testing
• [x] UX of all users considered:
  • Speech
  • Braille
  • Low Vision
  • Different web browsers
  • Localization in other languages / culture than English
• [x] API is compatible with existing add-ons.
• [x] Security precautions taken.

Summary by CodeRabbit

• New Features

  • Introduced an automated workflow for updating user documentation based on changes in markdown files, enhancing documentation consistency.

• Bug Fixes

  • Improved formatting and readability of various functions and classes across the codebase without altering functionality.

• Documentation

  • Enhanced clarity in comments and docstrings throughout the code, contributing to better maintainability.

• Chores

  • General code style improvements, including consistent spacing and formatting adjustments to adhere to coding standards.