CJK characters display "higher" than Latins, so the terminal scroll to wrong place after showing long CJK text

[cppHusky]

CJK characters in rio seem to displays "higher" than Latins. It's a good idea, because CJK characters become more readable with more vertical spaces. (In contrast, that in vim is less readable)

However, I found that after a long CJK text was output in rio terminal, it run into a strange status...

If all characters are Latin, everything works well like the top-left terminal shows.

But when characters are all CJK or CJK/Latin mixed, as the right terminal shows, the terminal scrolls less than 4 lines down after the printf command executed.

I think the problem is caused by rio assuming the height of contents is equal to that of 4 Latin character lines, not 4 CJK/Latin mixed character lines.

Here are two examples with two passages, one is Li Sao in Chinese, the other is a part of Hamlet. After cat Hamlet everything goes well, but after cat "Li Sao" it scrolls to quite a wrong place and I cannot even see the input line!

[Tryanks]

In the latest commit (191b5f9ad5c1615a61fb17a52e031ec3ff85b103) of the main branch, the line height of each line depends on whether the first line of characters in the current screen range includes CJK characters. If it does, the line height of the entire screen will be rendered at 1.1 times and break the lower boundary; otherwise, the line height will be rendered normally (whether it's a line of Latin characters or CJK characters).

It seems this issue was partially resolved in a certain change, but it's still not completely fixed. :(

[raphamorim]

Ok we are getting somewhere then.

[Tryanks]

Since I'm only learning Rust from scratch to investigate this issue, the following is just some slow progress I've made:
In the file sugarloaf/src/components/rich_text/mod.rs, when I modified the ascent and descent to fixed values, the issue described in this discussion was "fixed." ~This may not be a solution that should be adopted.~

So, here's my guess about what's really happening: When the font bundled with rio doesn't cover the characters appearing on the screen, it starts falling back to other fonts. And in the fallback fonts, it seems to pass a different font size, resulting in different measured ascent and descent values when rendering the glyphs.

I hope this helps with the progress of the issue, and I'll continue to work on it further.

rio is the first near-perfect solution I've found after struggling with Mac terminal issues for almost a year. I really like it and fully hope it becomes the most popular terminal.

[raphamorim]

To be frank I think we can actually use fixed values, would you mind doing a PR?

Originally it was written with round because of different line heights and would have some extra logic to match. But we don't need that anymore

[raphamorim]

rio is the first near-perfect solution I've found after struggling with Mac terminal issues for almost a year. I really like it and fully hope it becomes the most popular terminal.

Thank you for the message <3

[Tryanks]

To be frank I think we can actually use fixed values, would you mind doing a PR?

Originally it was written with round because of different line heights and would have some extra logic to match. But we don't need that anymore

I measured the actual values of the Metrics on my device, and the results for Latin and CJK characters are as follows:
Latin characters: ascent=25.976563, descent=6.5625, leading=0.0 CJK characters: ascent=29.927734, descent=7.5878906, leading=0.0

Therefore, I modified the ascent and descent values in the following two places to fixed values of 26f32 and 7f32:
https://github.com/raphamorim/rio/blob/c3d3640ab50668c3d3b532bc892c2ca725218415/sugarloaf/src/layout/render_data.rs#L126
https://github.com/raphamorim/rio/blob/c3d3640ab50668c3d3b532bc892c2ca725218415/sugarloaf/src/layout/render_data.rs#L188

// ...  
        let run_data = RunData {  
            span: style,  
            line,  
            size,  
            detailed_glyphs,  
            glyphs,  
            // ascent: metrics.ascent * span_data.line_spacing,  
            ascent: 26f32,  
            // descent: metrics.descent * span_data.line_spacing,  
            descent: 7f32,  
            // leading: metrics.leading * span_data.line_spacing,  
            leading: metrics.leading,  
// ...  

On my device, the issue appears to be completely resolved.
Did I do it correctly?

[raphamorim]

Thanks for the input, so what you did was match ascend and descend for both characters?

[Tryanks]

Different fonts return different ascent/descent values. For the default Chinese font "PingFang" on macOS, the OS/2 table lacks actual typographic values and falls back to the hhea table.
The win_ascender and win_descender values across fonts are very close but still differ by a few hundredths of a unit.

I'm genuinely stuck here. The issue objectively exists, and I can't propose a robust solution. The only workaround I have is to hardcode MetricsProxy with fixed values: ascent: 1900, descent: 480 (PingFang’s original hhea values are 2189, 555.)

Is this actually correct? I have no idea :(

https://github.com/raphamorim/rio/blob/cf5a52ebbdd72c4ba4607d4fade95c223784e7e4/sugarloaf/src/font_introspector/metrics.rs#L165

[Tryanks]

In the latest commit (191b5f9) of the main branch, the line height of each line depends on whether the first line of characters in the current screen range includes CJK characters. If it does, the line height of the entire screen will be rendered at 1.1 times and break the lower boundary; otherwise, the line height will be rendered normally (whether it's a line of Latin characters or CJK characters).

[img] It seems this issue was partially resolved in a certain change, but it's still not completely fixed. :(

Some progress: This behavioral change was introduced in commit f40379ce269fdfe9b74412e619cbb5d1141e44cf.

[raphamorim]

@Tryanks could you test with this branch? https://github.com/raphamorim/rio/pull/1208

[Tryanks]

@raphamorim Still get issue

clear && echo '测试Test'

[raphamorim]

thank you @Tryanks lemme double check

[raphamorim]

@Tryanks pushed few fixes https://github.com/raphamorim/rio/pull/1208 if doesn't work lemme know, trying to do an approach that can be outside of rendering operations