jquast
Add a test for compatibility workaround, U+2665 (♥) vs. U+2665 U+20 (♥ )
Apologies in advance if it's not possible or MacOS/Darwin isn't a target of your compatibility tests but otherwise:
https://github.com/ghostty-org/ghostty/discussions/7204
In every other terminal "♠♥♦♣" have the same rendering and width (single width) except with ghostty where the heart is special/uses a different font and is double width and hollow instead of what it's supposed to be (single width and filled)
Thanks! We do test:
- does expected narrow character become wide when followed by VS16?
but we can also test:
- does character defined as narrow, measure as narrow?
We can restrict this test to only the narrows defined by expected VS16 sequences without the VS16, thank you for your interest and idea
Of note that particular character width change in ghostty based on what follows it: (kitty top, ghostty bottom, same exact echo copy pasted)
echo "♠ ♥ ♦ ♣" "♠♥♦♣"
so to detect the issue you'd have to print it on its own and probably can't "see" the bug as the cursor is on the next one... so really just a bad font handling that halas is probably out of scope - I was hoping a bit of "score pressure" would help push this to resolution
Yes, unfortunately ucs-detect is blind of visual effects, a proper test would involve a user questionnaire, "do you see this, ? is it in color? is it free of obstructions? is this legible?" -- I will still do the additional narrow-stays-narrow testing proposed
I failed to mention the wcwidth-browser.py tool, which I made for developers to discover errors visually, here is a screen shot of Konsole with U+2665 at bottom-right, a snapshot of one out of 412 pages displaying East Asian Width text defined by category "Narrow":
In Konsole's case, they continue to render in their larger size and color presentation, but in Narrow cell representation as defined by East Asian Width, so it is sort of rendered "outside of its cell", whereas when displayed with FE0F (emoji style), it should be made wide as most modern terminals do, but Konsole also fails to report this as two cells, it shows almost the same results:
So in this case, we can see that both solutions by Konsole have undesired side effects that can be detected with this tool
In your "echo" example, you are adding spaces after each symbol, You can interactively insert space into a shell repl after each symbol to see them grow and shrink with ghostty. It is the spaces that causes it to change from "text presentation" to "emoji presentation".
I suspect this is a choice by a ghostty developer to make some form of compatibility for CLI applications that have inserted a workaround, of adding a "space" for the characters listed in emoji-variation-sequences.txt You can examine the source code and commit history of ghostty for this "spaced emoji" workaround to see if there is an explanation there.
To be brief, ghostty implements the unicode specification as correctly as I am able to measure, but additionally is adding a workaround, probably for compatibility of CLI applications or games, eg. card games, that were published in any of the past ~15 years these symbols were available but rendered incorrectly by most popular terminals (gnome, konsole, probably others), as pictured above.
It's really hard to this day, to reliably use either ♥ or ❤️ in terminals right now in a way that appears the same and consumes the same number of cells. One of them is popular for card games, and the other is probably among the top 10 most popular emojis, and they're both some of the earliest unicode symbols and emojis, so it is sad that they have gone unaddressed for so long. This is because they require a variation selector FE0F that is not supported.
I previously tried to help correct this in Gnome terminal by commenting on issue #2580 "Emoji / Unicode are not rendered correctly, regardless of font (incorrect width)"
It certainly makes for an even more challenging situation for Konsole and Gnome to have waited so long to support them drawn as narrow in text style, I wonder if they will make this backwards compatibility for themselves.
The codepoints you mention are displayed here on ghostty 1.2.3 as narrow glyphs, and they fit within their cell, and they are the "text style" -- bitonal and without any color, this is the correct interpretation of spec.
I will keep this issue open and revise the title -- should ucs-detect also test for this 'compatibility' workaround?
It might be useful to know...
well it's not just adding a space, it's rather that if there is another character after it shrinks; my actual use case is putting a ❤️ in tui playing cards and having all the other suits show fine (single width) and the heart is wrong (double width + hollow) it's placed on the screen 'by itself' (using ansi cursor move) and in that case it's "wrong" with now a terrible workaround consisting of adding an invisible 0 width space to shrink it back (which would fix 1/2 of the issue when the real issue is just they use the wrong/different font just for that 1 character)
in the above screenshot btw, it seems unless I read it wrong that behavior is consistent for all 4 symbols for that other terminal; which is all I need
In ucs-detect 2.0 I have released the wcwidth-browser tool to help visually detect these kinds of emojis (and wrote #27 about automatic check of visual results in some later release