Learn > Channels/Recommended - Khmer channel/content card description text spans beyond its container in Firefox and IE

[metodimilevqa]

Observed behavior

On the Learn > Channels tab a channel with Khmer language content has a description text that spans beyond its container. Experienced with Firefox and IE. On the Learn > Recommended tab some items with Khmer language content have a description text that spans beyond their containers. The issue is also seen in the View more section of the Recommended tab. Experienced with Firefox and IE.

Expected behavior

The card descriptions text must fit its container.

Steps to reproduce

Preconditions: Khan Academy (Khmer) channel imported. Kolibri language set to Khmer.

1. Login to Kolibri LP using Firefox or IE
2. Click the hamburger menu
3. Click Learn
4. See how the channel/items description renders in the Channels/Recommended tabs

Context

Kolibri 0.14.6rc1 Windows 7 vm Firefox 84.0.2 (32-bit) IE 11.0.9600

[jonboiser]

This looks related to #7750 and #5468 (and many others)

In Khmer orthography, spaces are not used between all words in phrases, which is why that description paragraph in the Khan Academy card has several long, unbroken strings. The original message in the code is also unbroken:

    Khan Academy ផ្តល់ជូននូវវីដេអូនិងលំហាត់គណិតវិទ្យា។ មុខវិជ្ជាទាំងអស់ត្រូវបានពន្យល់តាមរយៈវីដេអូដែលមានវិចារណញាណហើយមានលំហាត់ជាច្រើនដើម្បីជួយសិស្សឱ្យចេះស្ទាត់ជំនាញលើគំនិត។

And as noted in #5468, long strings like this can sometimes overflow their container. But it seems that there must be some mechanism browsers use to detect word boundaries in Khmer.

I'm thinking this might be an OS-dependent thing like lack of support for some opentype feature on Windows 7 if this isn't happening on newer OSs.

[metodimilevqa]

Just a detail - seems that Chrome handles it correctly in Windows 7

[jonboiser]

This might be a general problem with SE Asian languages. Here's a group that is investigating the issue

https://github.com/w3c/sealreq (see issue 34, which I'm not linking so it won't junk up their issue tracker).

This comment might lead to a possible solution:

The CSS spec says "Scripts such as Thai, Lao, and Khmer, however, do not use spaces or punctuation to separate words. Although the zero width space (U+200B) can be used as an explicit word delimiter in these scripts, this practice is not common. As a result, a lexical resource is needed to correctly identify soft wrap opportunities in such texts.", but doesn't provide any specific rules or properties for Lao line-breaking.

Basically, some browsers (like Chrome) might have a Khmer dictionary that helps them find words in these long unbroken strings. If that dictionary isn't available, it can use other hints like the zero-width space (which the KA Khmer description string doesn't have). Maybe this can be fixed on the Studio side and Crowdin to explicitly insert these spaces at word boundaries.

This is pretty laborious, but we can maybe test this by editing the locale file and adding U+200B characters at random places and seeing if Win7/FF will break at those parts of the message.

[jonboiser]

I just noticed on the shave.js page (the library that is supposed to handle the text truncation), there's an example of using shave in a non-spaced language. Maybe this example has some things we can try here. It just involves setting a {spaces: false} option.

https://codepen.io/yowainwright/pen/wzVgMp

[pcenov]

@radinamatic this is no longer valid in 0.15.1 - the Khmer text doesn't overflow both in IE11 and Firefox. Still, it's probably worth mentioning that there are some 0 like characters displayed in IE11:

[rtibbles]

@indirectlylit do you have any ideas about incorrect Khmer characters in IE11?

[indirectlylit]

Is the same behavior reproduced if the app language is switched to Khmer? I'm curious if it's missing characters only when the app language is something else.

IE11 ought to be using "basic" font-loading behaviors which are less efficient (no lazy-loading) but – if memory serves me correctly – ought to still be able to render all text correctly. Relatively recently, we noticed that IE was not behaving as expected and made this change: https://github.com/learningequality/kolibri/pull/8768

Here's the main branch point for testing. You might be able to reproduce this behavior in Chrome by forcing the first conditional to be false:

https://github.com/learningequality/kolibri/blob/1fd1fc08c92eae64634126a757c9e1af442870c2/kolibri/core/assets/src/utils/setupAndLoadFonts.js#L88-L94

[rtibbles]

The main issue here is fixed, and we are deprecating IE11, so I am closing this.