WIP use freetype for epaint text rendering

[lulitao1997]

implemented subpixel AA using freetype, this PR is working but still need to make freetype a optional dependency, maybe controled by a feature tag? related issue #2354

[emilk]

I would like to try this, but on my Mac I get:

error[E0308]: mismatched types
   --> crates/epaint/src/text/font.rs:96:63
    |
96  |                     freetype::ffi::FT_Get_Next_Char(raw_font, charcode, &mut self.gindex)
    |                     -------------------------------           ^^^^^^^^ expected `u64`, found `u32`
    |                     |
    |                     arguments to this function are incorrect
    |
note: function defined here
   --> /Users/emilk/.cargo/registry/src/github.com-1ecc6299db9ec823/freetype-sys-0.13.1/src/lib.rs:953:12
    |
953 |     pub fn FT_Get_Next_Char(face: FT_Face, char_code: FT_ULong, agindex: *mut FT_UInt) -> FT_ULong;
    |            ^^^^^^^^^^^^^^^^
help: you can convert a `u32` to a `u64`
    |
96  |                     freetype::ffi::FT_Get_Next_Char(raw_font, charcode.into(), &mut self.gindex)
    |                                                                       +++++++

I'd be far more interested in trying out https://github.com/pop-os/cosmic-text for the layouting and font rendering in egui.

[lulitao1997]

srry i don't have a mac, just pushed a commit that maybe fix this error. as for https://github.com/pop-os/cosmic-text, i didn't find lcd filtering in their code. Sadly i'm only able to get subpixel aa from freetype-rs, not from other pure rust libraries.

[LoganDark]

It is possible to perform subpixel antialiasing on the entire app and without freetype:

Demos

Subpixel AA

Grayscale AA

It works for all graphics, including widgets and vector drawings and even a 3D view or images:

3D painting

Subpixel AA

Grayscale AA

EasyMark Editor

Subpixel AA

Grayscale AA

Color test

Subpixel AA

Grayscale AA

[lulitao1997]

It is possible to perform subpixel antialiasing on the entire app and without freetype:

Demos It works for all graphics, including widgets and vector drawings and even a 3D view or images:

3D painting EasyMark Editor Color test

how do u do that?

[LoganDark]

The subpixel AA in this post, and above, accidentally assume a BGR panel instead of RGB, oops. It is trivially reversible, and the same concepts still apply

how do u do that?

After 14 hours of searching I can't seem to find the article that I read that explains subpixel antialiasing. But basically, you can apply it to an entire application in the exact same way that you would apply it to text.

The basic idea is to render to a buffer that is three times as wide as the actual window, and then downsample it using a subpixel antialiasing filter. This can be done efficiently with a fragment shader on the GPU, and doesn't require switching the entire text rasterization backend (esp. when FreeType would be one more non-Rust dependency to build).

Let me explain in more detail, since that blog post is unfortunately lost to my browsers' lack of good history search.

Let's say you are trying to subpixel-antialias a lowercase e. Here is a close-up:

The point of subpixel antialiasing is to approximate the shape better, like this:

We want to find a way to use the full precision of the LCD's subpixels to achieve this, since they can theoretically achieve a horizontal resolution of three times what grayscale AA can.

First, though, we need to create a rendering where what will become each subpixel is represented individually. This can be done by rendering it at 3x width:

Each three columns represents what will become only one column of output. The not-quite-right approach is to try to combine them as-is - that is, for each set of three columns, take the red channel from the left column, the green channel from the center column, and the blue channel from the right column. (And vice versa for BGR panels)

This results in:

But this is obviously too intense - for example, there is bright yellow to the left side. We're only missing one step, though.

The trick is to average all pixels with their horizontal neighbors, so that the color fringing is less pronounced. That results in this rendering:

and this subpixel-antialiased result:

This approach is very easily generalizable - it can be applied to much more than just text. Here it is applied to this GitHub issue:

Notice how not only is the text subpixel-antialiased, but everything is - border radii, emoji, your profile picture, icons and solid shapes.

However, this reveals an issue with my implementation - solid pixels bordered by other solid pixels bleed into each other by one subpixel. I'm sure this can be fixed somehow - and I've tried not taking into account neighboring full-pixels - but I just can't seem to get a result as good as what I've shown so far.

As always, more research is needed.

(It took me four hours to compose this comment lol)

[lulitao1997]

The subpixel AA in this post, and above, accidentally assume a BGR panel instead of RGB, oops. It is trivially reversible, and the same concepts still apply

how do u do that?

After 14 hours of searching I can't seem to find the article that I read that explains subpixel antialiasing. But basically, you can apply it to an entire application in the exact same way that you would apply it to text.

The basic idea is to render to a buffer that is three times as wide as the actual window, and then downsample it using a subpixel antialiasing filter. This can be done efficiently with a fragment shader on the GPU, and doesn't require switching the entire text rasterization backend (esp. when FreeType would be one more non-Rust dependency to build).

Let me explain in more detail, since that blog post is unfortunately lost to my browsers' lack of good history search.

Let's say you are trying to subpixel-antialias a lowercase e. Here is a close-up:

The point of subpixel antialiasing is to approximate the shape better, like this:

We want to find a way to use the full precision of the LCD's subpixels to achieve this, since they can theoretically achieve a horizontal resolution of three times what grayscale AA can.

First, though, we need to create a rendering where what will become each subpixel is represented individually. This can be done by rendering it at 3x width:

Each three columns represents what will become only one column of output. The not-quite-right approach is to try to combine them as-is - that is, for each set of three columns, take the red channel from the left column, the green channel from the center column, and the blue channel from the right column. (And vice versa for BGR panels)

This results in:

But this is obviously too intense - for example, there is bright yellow to the left side. We're only missing one step, though.

The trick is to average all pixels with their horizontal neighbors, so that the color fringing is less pronounced. That results in this rendering:

and this subpixel-antialiased result:

This approach is very easily generalizable - it can be applied to much more than just text. Here it is applied to this GitHub issue:

Notice how not only is the text subpixel-antialiased, but everything is - border radii, emoji, your profile picture, icons and solid shapes.

However, this reveals an issue with my implementation - solid pixels bordered by other solid pixels bleed into each other by one subpixel. I'm sure this can be fixed somehow - and I've tried not taking into account neighboring full-pixels - but I just can't seem to get a result as good as what I've shown so far.

As always, more research is needed.

(It took me four hours to compose this comment lol)

thanks so much for the detailed reply!