Chinese font support

[kekearif]

Is there a way to use Chinese font in iosMath? currently if I try to include Chinese characters in an equation it just skips them and doesn't display anything.

[AnZhg]

@kekearif I believe Chinese characters are skipped because the provided fonts DO NOT include any Chinese character. You can check it using any font viewing apps. After you add the missing fonts, I think you can display Chinese characters without modifying any source code. And you can always add your own commands/symbols in MTMathAtomFactory.m, + (NSMutableDictionary *)supportedLatexSymbols.

[kostub]

Chinese font is difficult to support because LaTeX is inherently ascii. Any non-ascii characters are excluded by default. There are some extensions to LaTeX which have unicode support, but that has not been implemented and will require substantial effort.

On Mon, Jan 2, 2017 at 10:50 PM, AN Zhigang [email protected] wrote:

@kekearif https://github.com/kekearif I believe Chinese characters are skipped because the provided fonts DO NOT include any Chinese character. You can check it using any font viewing apps. After you add the missing fonts, I think you can display Chinese characters without modifying any source code. And you can always add "commands" in MTMathAtomFactory.m, + (NSMutableDictionary *)supportedLatexSymbols.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kostub/iosMath/issues/27#issuecomment-270059371, or mute the thread https://github.com/notifications/unsubscribe-auth/AAFeXB_3hKQ6M2994VYXol8oKKpOtI5tks5rOe_JgaJpZM4LYFow .

[kekearif]

@kostub it wouldn't work if I just added a Chinese font as described by @AnZhg above?

[AnZhg]

@kekearif Adding Chinese font would NOT work, at least not easily. I don't know how pdfLaTeX and xeCJK implement limited simplified Chinese support.

But if you want to support only limited number (say, 20) of Chinese characters/phrases, the second method I mentioned above works. Here is the result I have:

I'm using otf format of Hiragino Sans GB (冬青黑体) W3 font. Manually add a command:

@"Chinese" : [MTMathAtom atomWithType:kMTMathAtomOrdinary value:@"中文"],

Use it:

self.label.latex = @"\\Chinese";

[AnZhg]

@kekearif Ignore my last comment, it works. After diving into the code, I found in file MTMathAtomFactory.m, method + (MTMathAtom *)atomForCharacter:(unichar)ch, parser ignores any character out of the Unicode range < 0x21 or > 0x7E. To get CJK support (though still limited Chinese characters, and Japanese, Korean included), edit the following lines:

if (ch < 0x21 || ch > 0x7E) {
    // skip non ascii characters and spaces
    return nil;
}

into:

if ((ch >= 0x4E00) && (ch <= 0x9FFF)) {
    // CJK support.
    return [MTMathAtom atomWithType:kMTMathAtomOrdinary value:chStr];
} else if (ch < 0x21 || ch > 0x7E) {
    // skip non ascii characters and spaces
    return nil;
}

And now, everything works. But remember to add missing CJK fonts (Unicode range, 0x4E00 - 0x9FFF, inclusive).

Here is a screenshot:

WARNING, DISCLAIMER: I don't know if there is any side effect, not extensively tested.

@kostub Maybe this feature can be added to iosMath? But the included fonts have to be modified. I don't know if the font licenses allow distribution after modification.

[AnZhg]

@kekearif I just issued a pull request which added Chinese feature to iosMath. Check out this pull request.

[kekearif]

@AnZhg This is amazing! Thank you so much for your work!

[AnZhg]

@kekearif Yep. @kostub makes a great project.

[kekearif]

@AnZhg I was afraid I was going to have to use MathJax in a web view. Now I just gotta figure out getting the Chinese font rendering in the {text} label https://github.com/kostub/iosMath/pull/19. ⛽️

[AnZhg]

@kekearif But why that pull request closed by its issuer?

MathJax is way too slow. I would not use it on any iOS device. I was using MathJax, and I had to add a image layer so that 20 ms rendering delay won't affect UI refresh.

[kekearif]

@AnZhg not sure why it was closed. But it works and it's what I'm using. Yeah if I was forced into using MathJax I'd also hide the main view while it was rendering.

[kostub]

Thanks for the PR @AnZhg. It will be a great addition. I've suggested some changes on the PR.

[wingsiu]

How about Emoji?

[AnZhg]

@wingsiu I don't think this is an easy task. Since all fonts/characters are rendered by bezier path, and fill the path with one given font color (default is black), while emoji's are colorful pictures. But, do we really need emoji support? Symbols like "" (Apple symbol, if you are not using Mac OS/iOS), would be pretty easy, just like Chinese characters, these symbols are just an Unicode(though the apple symbol is in private region).

[kostub]

@AnZhg @wingsiu Shouldn't Emoji work similarly? Emoji characters are just unicode characters, so as long as there is a font has supports the emoji characters they should be able to be rendered the same way as the Chinese characters are.

[AnZhg]

@kostub If you want a monochrome version, or the plain version, I don't think there would be any problem. But I don't see any code that handles colorful glyph, just using fontColor to render all bezier path. So, I think the result would be single color rendering version, instead of what we see on iOS device/Mac. That is, ❤ will render as ❤\U0000FE0E.

[AnZhg]

Oh no.... Mac OS automatically substitute the emoji... And I cannot force substitution off... Anyway, this link may be useful (to "see the plain version"): https://paulofierro.com/blog/2013/3/30/disabling-emoji-characters-with-unicode-variants

[kostub]

Fonts are not rendered using bezier path, fonts are rendered using CoreText which is the underlying text formatting for all iOS apps. Would CoreText not rendered the colorized versions as long as they are present in the font?

[AnZhg]

@kostub Okay, then we need loading fonts from multiple files support. Colorful version is from Apple Colorful Emoji font file, and I don't think it is legal to copy those emojis into the open fonts.

So bezier path is only used to render non-text elements? (e.g. root symbol)

[kostub]

So bezier path is only used to render non-text elements? (e.g. root symbol)

Only the lines - fraction line, the top of the square root, underline, overline etc. Even the root symbol is from the font files.

I don't think it is legal to copy those emojis into the open fonts.

Yes. I'd rather have an API that people can use to add their own Chinese or Emoji or any other language font rather than having to build open fonts that contain that data.

[wingsiu]

I can input emoji without adding the fonts now. Can i not adding the fonts? will any problems?

[kekearif]

@AnZhg do you know if it's possible to use a different type of Chinese font? I'm wanting to use Arial Unicode MS. How exactly did you create the font file? 新年快樂！

[AnZhg]

@kekearif It is possible to use a different font, any font. I was using a trial version of Glyphs on Mac. I tried GNU Font Editor, which is free but hard to use, requires X11, and has a non-retina resolution. Glyphs.app is faster, and more stable, but still would crash if you copy/paste too many glyphs. I think there are better softwares on Windows.

Let's assume you have a working font editor. And Arial Unicode MS has Chinese characters in it. All you need to do is copy glyphs in Unicode range 0x4E00 ~ 0x9FFF, inclusive, and then paste them in the font you want to add Chinese support to. This range is CJK range, which means some of the characters are Japanese, or Korean, Others are Chinese, both simplified and traditional. It's hard to separate these three languages apart, because, as far as I know, Chinese and Japanese share a lot of glyphs. They have the same character. Also, Chinese simplified and traditional share the same characters heavily. You may need to copy glyphs several times, such as from 0x4E00 ~ 0x5DFF, 0x5E00 ~ 0x6DFF, ... I did this, or Glyphs would halt the whole system.

Note that this is not the whole Chinese character table, there are some extensions in other Unicode range for Chinese, but is enough for daily use.

Another note, if the font comes with different sizes in different files, it is better to try to copy/paste one glyph first. The glyph of certain size may be too large or too small for the font file you want to modify.

Last, 新年快乐！

[AnZhg]

@kekearif At some time, I will try to write a better Chinese support for iosMath. Maybe let user specify which font file to use programmatically. But for now, I have to finish my thesis paper first, and find a job. Finding a job in US is hard for an international student, maybe even harder after the new policy.