Scalable SDF fonts & shadows

[bvgastel]

This pull requests adds scalable fonts to Dear ImGui. A technique to implement scalable fonts is using signed distance fields: make a bitmap where every pixel value represents the distance to the edge of a shape/font. Values [0, 0.5] represent distances outside the shape, values [0.5, 1] inside the shape. A value of 0.5 is on the edge of the shape. If this bitmap is scaled up these values are automatically interpolated. By using a shader, these interpolated values can be turned into sharp edges, avoiding the normal blurriness if scaling fonts up.

When adding a font, an additional flag can be set in ImFontConfig, called SignedDistanceFont. If set this flag causes imstb_truetype to load the font as a signed distance field sized (currently) 40 pixels (IMGUI_SDF_DETAILS define) with padding 4 (IMGUI_SDF_PADDING define). This increases the font atlas a bit, but yields very good results when rendering, with almost no artifacts. When rendering fonts, the extended shader is automatically used if available. If the shader is not available (for example with OpenGL 2), the signed distance rendering (and font loading) is disabled.

To indicate support, backends can now set an additional flag ImGuiBackendFlags_SignedDistanceFonts. The backend Init() methods are extended with an additional field, in which the programmer can indicate which features it wants to be enabled. The default value is ImGuiBackendFlags_DefaultFast, which currently disables signed distance fonts, so current users are not confronted with surprises (with increased font atlas size, and a bit longer loading time). ImGuiBackendFlags_DefaultDesktop enables the signed distance features.

Signed distance fields also allow certain text effects such as shadows and outlines. To support this, ImGuiStyleVar_FontShadowSize indicates the size of the shadow/outline, and the colors ImGuiCol_FontShadowStart and ImGuiCol_FontShadowEnd specifies the start and end colors of the outline. If these colors are the same, it is an outline. If the alpha value of the end value is set to transparent, it is a shadow.

To support these new font loading, the vertices generated are extended with additional fields: (1) a start color, (2) an end color, (3) a threshold called a signifying where the inner color stops, and the start outer color starts (a float), (4) a threshold called b signifying where the outer color ends (a float), (5) for anti-aliasing a width called w (a float). This makes the vertices list twice as large, increasing an individual vertex from 20 bytes to 40 bytes. This is clearly a drawback of the current approach. An alternative approach could be to encode the end color with only an alpha value (and using the same values for the color channels as the start color), and using unsigned shorts for the a, b, and w values. This will result in a vertex size of 32 bytes, but with somewhat less flexibility (and maybe encoding problems in some shader languages?).

To compensate the increased vertices size, another route was taken. If the a value is set between 2 and 3, it is interpreted as a distance request for a quarter of a circle. UV values (0,0) are interpreted as the center of the circle, (1, 0) and (0, 1) are on the edge of the circle. This cost no additional size in the vertices, but allows for perfect shadows of shapes. A rounded rectangle with shadow can be drawn with 18 triangles (size of shadow or rounding does not have an impact). See the DrawCmd in Metrics/Debugger window how these rectangles are drawn (especially look for shapes ones with some round corners, and some straight corners). In the demonstrator, the same screen using regular Dear ImGui was drawn using 15734 vertices, 37899 indices, and 12633 triangles. Using the change in this pull request, the same scene was drawn using 11052 vertices, 19953 indices, and 6651 triangles. See the attached screenshot. In these screenshot, the windows and frames have a border (in the regular version), or a shadow (if using this patch).

In the draw list, AddRect and AddRectFilled are changed to use the new signed distance shape rendering if available in the backend (which indicates support using the ImGuiBackendFlags_SignedDistanceShapes). If not available, these methods use the existing drawing primitives. Some additional arguments are added to these methods, but use sensible defaults, so existing code keeps working with no change required. The frame and window rendering code was changed, so frame and window shadows are added. To allow easy styling, the following variables are introduced: ImGuiStyleVar_WindowShadowSize, ImGuiCol_WindowShadowStart, ImGuiCol_WindowShadowEnd, ImGuiStyleVar_FrameShadowSize, ImGuiCol_FrameShadowStart, ImGuiCol_FrameShadowEnd. The new extra argument to the backend Init() method can be used to enable or the fonts feature, or the shape feature, both, or none.

If there is a need to completely disable these signed distance features, a new flag IMGUI_DISABLE_SDF can be set in imconfig.h complete disabling all the signed distance fonts. To aid testing these features, a demonstration is made in the GLFW OpenGL 3 and the DirectX 11 examples, in which all fonts are loaded twice: as signed distance and regular. Attached are screenshots of two scenes of this demonstration. Each scene is rendered twice: with the signed distance features enabled and disabled (using exactly the same code except for the IMGUI_DISABLE_SDF define).

Special attention has been paid to smaller fonts. Both ProggyClean and ProggyTiny are rendered exactly the same with or without signed distance. Due to the way imstb_truetype calculates signed distance fields, some fonts will be rendered one pixel higher (as is the case with ProggyClean). Roboto, Cousine, Droid Sans are rendered similarly on 16/15 pixels when rendered with or without signed distance fields.

The shaders are currently implemented for:

• OpenGL3 shaders (except for GL SL version 120) (tested on Linux);
• Metal (tested on macOS);
• DirectX 11 (tested on Windows 10).

A logical question is probably why multichannel signed distance fields were not used. This makes loading of the fonts complex. If a live encoding is used, this makes the loading of the fonts slow. A prefab multichannel font atlas was found to be large, and loading cumbersome. Another strategy is to cache the generated multichannel font atlas. I have experimented with that, but was not satisfied with the result (version specific custom storage format, problems with invalidating the cache, etc).

I'm currently looking for feedback on how to improve this pull request. Before investing more work, I would appreciate a feasibility check if this pull request has chances to be merged at all. If that is the case, I can pick up the remaining issues, such as:

• implement the new shaders on DirectX 9, DirectX 10, DirectX 12, and Vulkan;
• try more fonts, and fix any alignment issues (such as the 1 pixel different with the ProggyClean font described above);
• writing documentation;
• look into supporting the freetype font backend.

The screenshot of the first scene, with SDF: and without SDF: .

The screenshot of the second scene with SDF: and without SDF: .

[SadE54]

Really nice work !

[PathogenDavid]

I didn't dig into anything super closely yet, but the results certainly look nice!

A few random questions:

• Do the newly-added vertex components actually vary between glyphs in the same render call? Could they just live in a constant buffer which is changed by a draw command?
• Does this play nice with colorful icons?
• It would probably preclude using SDF for window edges and such, but did you explore what a pluggable font rendering system might look like? (IE: Rather than having Dear ImGui know anything about SDF, you might load something like a "D3D11 SDF font rendering backend" which handles all the SDF concerns there.)

---

and a bit longer loading time

It takes quite a lot more than "a bit" on my machine. Here's some performance metrics I gathered from the DX11 sample. All times are in milliseconds. (Atlas A is the font atlas in this PR. Atlas B is with only loading GetGlyphRangesJapanese+鑰 from ArialUni.ttf @ 16px. SDF was toggled by changing the flag passed to ImGui_ImplDX11_Init and ImFontConfig::SignedDistanceFont)

Action	SDF Off	SDF On
Vertex shader compilation	8.10	12.44
Pixel shader compilation	5.18	28.57
GetTexDataAsRGBA32 A	213.74	4,703.98
GetTexDataAsRGBA32 B	242.97	16,679.96

Assuming the texture generation can't be improved: In my own backend I'd probably rework things to generate a non-SDF font atlas first and regenerate a separate SDF font atlas on a background thread and swap it in once it's finished. I'm not sure if that level of complexity is something we'd want in the example backends though. Another option would be to cache the font atlas on the disk, but that still seems not-ideal for the examples.

---

Also for those like me who are curious to see how it handles more complex glyphs, here's some screenshots with it rendering "鑰 日本語" using Arial. (鑰 is by no means common, I just grabbed a random kanji that was on the more complex end.)

SDF on:

SDF off:

[bvgastel]

Thanks for looking into this. I have used Font Awesome 5 while developing this feature, never tested it with Japanese fonts. Nice to see that screenshot. Here are some Font Awesome 5 icons:

Colorful icons

Does this play nice with colorful icons?

Custom colorful icons (and other things packed in the font atlas) should just work as normal, no special handling is needed. A different shader path is activated. There are three modes: (1) a = 0.0 which activates the traditional shader path (using color hinting); (2) a > 0.0 and a <= 1.0 which activates the SDF fonts mode; (3) a >= 2.0 and a <= 3.0. All the draw calls have new arguments with default values that use the first mode: the traditional shader code.

Vertex components

Do the newly-added vertex components actually vary between glyphs in the same render call? Could they just live in a constant buffer which is changed by a draw command?

Yes. Depending on the amount of shadow and color wanted, these vertex components vary. The b component specifies how large the shadow is, so can vary per glyph. Likewise, per glyph the color can change: the start and end color specify the color of the shadow on the border with the inner color and on the brink of where the shadow stops. The w component is needed for proper anti-aliasing, and specifies the width of anti-aliasing. For text, the a component is always 0.5, however for regular font atlas drawing calls (like lines), and other textures (e.g. images), this a component can be set to 0.0. We can set the a component to 0.48 to add a fake bold option. And this a component is also used for the shapes feature.

I'm assuming we want to batch draw calls aggressively. We can batch it differently: every time the text style changes (i.e. shadow size, shadow color), create a new draw command. I'm suspecting that the size of the ImDrawVert struct is the source of this question. If that is a problem we can try to reduce this by using different types for all the extra fields (see original PR text), reducing the struct from 40 bytes to 32 bytes (currently it is 20 bytes).

Pluggable rendering?

It would probably preclude using SDF for window edges and such, but did you explore what a pluggable font rendering system might look like? (IE: Rather than having Dear ImGui know anything about SDF, you might load something like a "D3D11 SDF font rendering backend" which handles all the SDF concerns there.)

I have not tried it. I imagine that this would make the change a lot larger. The large chunk of the font system should also be moved to this pluggable system. The render backend has to know if a glyph is in signed distance or not, this should be stored somewhere. It would probably either hurt the batching (as fonts are rendered differently from lines and the other icons in the font atlas), or loses the ability to mix signed distance fonts and regular fonts.

Timings

About the timing, generating the SDF takes indeed significant more time per glyph. My test case was with a limited number of glyphs: a roman character set with some glyphs of Font Awesome, and the roman character set of the included fonts. Some timings (Linux, release build):

Font	regular	SDF
Roboto Medium	2ms	91ms
Font Awesome 5	32ms	4870ms
Fonts in the SDF example	31ms	1177ms

As mentioned in the original PR, I have the code to save the font atlas to disk, and reload it from disk. Generating this font atlas file beforehand could work in some cases, but probably not all. Caching could be a viable option: first start generate the font atlas cache file, and update if necessary. I'm not sure about this solution, due to the added complexity and need for proper invalidation (if the font file changes, the required texture format, etc). It is very fast though.

The calls to stbtt_GetGlyphSDF2 can be executed in parallel, so there are threading opportunities. Not good for the examples, I suspect, but for my own versions I would probably go for that.

[PathogenDavid]

Thanks for the info!

Vertex components

Yes. Depending on the amount of shadow and color wanted, these vertex components vary.

To clarify, I was referring to a single call to ImDrawList::AddText. (Or a series of text draws where all of the text has the same style.)

I'm assuming we want to batch draw calls aggressively. We can batch it differently: every time the text style changes (i.e. shadow size, shadow color), create a new draw command.

This is more or less what I was getting at. At a glance it feels like storing these parameters per-vertex is optimizing for the case where these parameters change frequently (which seems less likely to me), but I also wasn't thinking about them needing to change between text and shapes so that's probably more reasonable than my initial reaction.

(I'm also looking at this from the perspective as someone who is mainly here for the nice-by-default font rendering on per-monitor DPI systems.)

Pluggable rendering?

I have not tried it. I imagine that this would make the change a lot larger.

For sure, I was just curious if you explored it at all since it's something I've wanted to explore myself.

Timings

As mentioned in the original PR, I have the code to save the font atlas to disk, and reload it from disk.

Ah, I missed that bit on the re-read apparently.

My main concern is that enabling SDF in the examples by default seems like a good idea for the sake of demonstrating it, but that it'll give people a bad first impression of Dear ImGui if it makes things load so slowly.

Or if performance in the examples is good enough once it's only loading a single font with only , CJK users might feel duped when startup performance suddenly tanks in response to enabling their character sets.

Not good for the examples, I suspect, but for my own versions I would probably go for that.

I agree, but I'm also under the impression that a lot of people only ever use the backends provided in this repository. (But that might also just be because people who can write their own backend are less likely to ever need to ask questions.)

[bvgastel]

Thanks for looking into this PR. I think reducing the ImDrawVert struct to 32 bytes is a good middle ground here. It is still flexible, a limited increase from the regular Dear ImGui, and no need to add logic to separate all the text draw calls from the shape draw calls.

This scalable font rendering is especially useful for per-monitor DPI, that is a nice use case I did not realize. How are you adjusting for the dpi? Using the window/global font scale?

About the timings, and slowness. There is a comment in the stb code, that there is an optimization TODO: line 4490 of `imstb_truetype.h. But I never dived into how fonts are stored in font files, and how to use them. I'm not sure if I can contribute there.

If we don't explore that TODO, it boils down to pregenerating/caching vs threading, or both. Caching seems to be the best option. Do you expect that would be general acceptable?

[meshula]

Love this!

+1 for caching, and explicit save/load.

[bvgastel]

I did some timing tests on the stbtt_GetGlyphSDF2 function. That TODO on line 4490 turned out to be insignificant. However, I switched a couple of loops, eliminated an allocation in the progress. Reduced the runtime for signed distance generation in one example (Roboto + Font Awesome 5) from 4.9 seconds to 3.0 seconds (different computer than the above one, although comparable result). That is almost saving 40% of the runtime. I will clean up the code and recheck if I made any errors before committing (this was a feasibility check).

That will not be enough for all users (so we still have to look into alternatives), but I'm happy with this result nevertheless.

@PathogenDavid for your timing measurements, did you use a debug or a release build?

[PathogenDavid]

This scalable font rendering is especially useful for per-monitor DPI, that is a nice use case I did not realize. How are you adjusting for the dpi? Using the window/global font scale?

I'm on the docking branch, so I use ImGuiConfigFlags_DpiEnableScaleFonts which scales the fonts based on the viewport's DPI scale. It gets the job done, but it's not pretty.

If we don't explore that TODO, it boils down to pregenerating/caching vs threading, or both. Caching seems to be the best option. Do you expect that would be general acceptable?

That's a question for Omar since they have different tradeoffs as far as the examples and first time Dear ImGui user experience is concerned.

In my backend I'm inclined to implement it as generating the non-SDF font atlas first, loading the SDF version in the background, and swapping it out before ImGui::NewFrame once it's loaded. (And if the effect from the text jumping around slightly when the SDF font loads is annoying, add caching.)

Reduced the runtime for signed distance generation in one example (Roboto + Font Awesome 5) from 4.9 seconds to 3.0 seconds

Thanks for looking into that! That's a pretty significant win.

for your timing measurements, did you use a debug or a release build?

I just opened the Visual Studio solution and ran example_win32_directx11 without changing anything, so that would've been Debug x86 MSVC v110 (2012), so not quite the ideal situation for performance.

My day to day usage of Dear ImGui is Debug x64 MSVC v142 (2019), which I'm sure is at least a little bit better but I haven't has a chance to try this branch with real software yet. (Although either way, that's not the default fresh clone experience.)

[ocornut]

Hello,

Thanks @bvgastel for that work. I won't have time to dig into this soon but FYI it is expected we switch to a modal where glyphes will be rasterized on demand, so initial build time isn't so much of an issue there. That work will be done likely before we even look at SDF work (first step is #3761, second step glyphs on demand which will require a non-trivial refactor of font access and packing logic, third step may be to adopt something like this SDF thing or another solution. Note that second step will technically have the side-effect of mitigating the pressure for SDF-like rendering, but of course if it can be plugged in and useful it'll be good).

I noticed you added flags to backend Init functions, AFAIK those can be something like ImGuiConfigFlags_EnableSdfFonts ImGuiConfigFlags_EnableSdfShapes and then those config flags gets cleared by dear imgui if the required backend flag isn't available. Similarly, the rendering code ends up testing those config flags (not the backend flags which are more like "capabilities" and won't change after init).

[bvgastel]

@ocornut: Thanks for letting me know. Seems that this PR is a long way out. Honestly I don't know if it is worth looking at SDF fonts after on-demand glyph loading is there, which can easily be used to support text at different scales. Adding this PR adds complexity and long run maintenance costs. Advantages that remain for the general user population are perfect and low cost shadows, but that could be a separate PR/feature. Dynamic glyph loading should of course be fast, to avoid stalls. If not, SDF can help there. But I have faith in the speed of dynamic glyph loading.

@PathogenDavid: I couldn't resist, and did look into threading yesterday. For which I think I have an elegant solution, that is flexible for multiple platforms (about 20 lines platform specific code, and about 30 lines of generic code). This reduces the load time from 3.0 seconds to 0.6 seconds.

[bvgastel]

As promised, I cleaned up the SDF generation speed boost. Single threaded (Roboto + Font Awesome 5) runtime is down from original 4.9 seconds, to 3.0 seconds (as reported above), to 2.1 seconds, so it uses 43% of the runtime of the original version. Multi-threaded version calculates the same in 428 ms (on my 6 core laptop).

I'm still not really sure how to progress, and if it will be useful in the end. So I'm blocked right now on this feature.

For future discussions, to do:

• resolve if adjusting ImDrawVert and adding shaders is ok at all;
• decide if reducing the ImDrawVert size is needed;
• resolve if SDF features is a backend capabilities flag or global config flag;
• original todo (about backends, freetype, etc).

[rollraw]

btw related to that, some days ago freetype 2.11.0 was released within builtin sdf support

[bvgastel]

I have done some work to reduce the ImDrawVert from 40 bytes to 32 bytes (mainline is 20 bytes), now only for OpenGL 3. If somebody is interested in this, let me know.

[frink]

Any hope of a merge yet?

[cloud11665]

Superb work! I myself have been thinking about integrating SDF based fonts into my application. My idea was to use https://github.com/Chlumsky/msdfgen and just prompt the user with "generating fonts" modal upon first launch, and just load the whole font atlas from a disk cached image on later launches.

[ryuukk]

Found this repo, someone added sdf support for freetype's backend https://github.com/lliryk/ImGui-SDF/