gf icon indicating copy to clipboard operation
gf copied to clipboard

Add basic Unicode support

Open MadCatX opened this issue 1 year ago • 8 comments

This patch adds basic Unicode support to the UI. It relies on Freetype font renderer and is hidden by default behind UI_UNICODE flag. Encoding support is currently limited to UTF-8.

What works:

  • Display of all Unicode characters. Tested against (https://github.com/bits/UTF-8-Unicode-Test-Documents/blob/master/UTF-8_sequence_separated/utf8_sequence_0-0x10ffff_assigned_including-unprintable-asis.txt) and (https://github.com/bits/UTF-8-Unicode-Test-Documents/blob/master/UTF-8_sequence_separated/utf8_sequence_0-0x10ffff_assigned_including-unprintable-replaced.txt)
  • Handling of invalid UTF-8 input. Invalid characters are replaced with ? without any other adverse effects.
  • Navigation in text using keyboard and mouse, selection, copy-paste, replacements.
  • Starting executables from paths with Unicode characters. The path currently needs to be pasted into the textbox.

What could be improved:

  • Word-skipping in text with Unicode characters is a bit flaky. To fix this, I suppose that the IsCharAlpha() function would have to be much more sophisticated.
  • Cursor position selection with mouse and invalid UTF-8 input. Sometimes the cursor is positioned a few characters behind the expected position if the expected position lies inside an invalid UTF-8 sequence. Navigation with keyboard seems to work fine.

What does not work:

  • Textboxes currently do not accept Unicode input from keyboard keystrokes. A workaround is to copy-paste it in.

Notes:

  • This patch changes the meaning of carets in UIStringSelection to by how many glyphs the caret is offset from the beginning rather than bytes.
  • This patch also fixes a minor rendering bug when a text selection in UITextbox contains tabs.

Comments welcome :)

MadCatX avatar Feb 25 '24 18:02 MadCatX