HarfBuzz applies morx as part of AAT, which in some fonts differs from GSUB and isn’t applied on Windows

[ghost]

(Updated) Currently, some ligatures in some fonts are not taken, although they work in other applications like Windows font preview.

This is related.

Workaround: Open the font with FontForge, scroll to the last row, for each glyph without a code point: assign one starting at for example 100.000. After this, it will work flawless. Updates to the ligature tables are not needed, since the glyphs before where adressed by name not by value.

A first solution to this could be to print out a warning to the console (easiest approach: if at least one glyph exists that has no unicode value assigned, better approach: if at least one glyph exists that has no unicode value assigned and is refered to in the liga table: warn), took me some time to find out what was the problem.

[astiob]

Please provide a full sample that fails.

Ligatures without code points work fine in the first font I tried, Cambria Bold (ffb → unmapped ligature glyph), and even the issue you link references Zapfino as an example of a font with lots of such ligatures that work just fine.

[ghost]

You're right, the Cambria TTF without unicode code points seem to work. I'll try to find the specific thing that is broken, maybe it's only with OTF-fonts. I will add an example later.

[ghost]

Most fonts seem to work in both TTF and OTF, I could not check the Zapfino, I think it's not freely available. But here is another one by Zapf and it does not work as expected, it does work correctly on Windows font preview, though. The expected ligature in word test is "longs t" using this font. When I find more fonts that can be uploaded or linked according to the license and don't work, I add them here.

[Script Info]
; Script generated by Aegisub 3.2.2
; http://www.aegisub.org/
Title: Default Aegisub file
ScriptType: v4.00+
WrapStyle: 0
ScaledBorderAndShadow: yes
YCbCr Matrix: None
PlayResX: 1280
PlayResY: 720
Kerning: yes

[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
Style: Default,LOV.Gilgengart,90,&H00FF000F,&H00FFFFFF,&H00000000,&H00000000,0,0,0,0,100,100,0,0,1,1,1,5,100,100,20,1

[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,0:00:00.00,1:11:11.11,Default,,0,0,0,,test

[astiob]

Thanks! However, I cannot produce the ligature you mention with the given font in hb-shape, either, so at the very least this doesn’t seem to be a libass-specific issue. I can produce the same ligature after simply re-saving the font from FontForge without any changes at all, so I suspect this is either a bug in the font itself (which FontForge helpfully fixes automatically) or a bug in FontForge. Unfortunately, this font’s OpenType lookups, as shown to me by FontForge, are too complex for my own understanding (and, frankly, suspicious). But what I can tell using fonttools ttx is that the FontForge-generated file contains a whole lot of GSUB lookups that are absent from the original file.

[khaledhosny]

The font contains both GSUB table (for OpenType layout) and morx (for AAT layout), and it seems the morx table does not implement all substitutions in the GSUB table (the font was generated by FontForge, and FontForge has an option to automatically create AAT tables from OpenType ones, but it is a broken feature and most likely this is what generated the morx table).

HarfBuzz supports both OpenType and AAT and picks morx over GSUB. Uniscribe/DirectWrite do not support AAT so the morx table is ignored and GSUB is used in Windows font viewer. If I removed the morx table (which probably what happens for you when you open and generate the font with FontForge), I get the long s substitution. Apparently this font was never tested on macOS, since CoreText behaves the same as HarfBuzz.

Overall, it is all the fonts fault here, but if replicating Windows behavior is a must, you probably need to stop sending AAT tables to HarfBuzz when it asks for them, since it currently has no API to prefer OpenType layout over AAT one.

[khaledhosny]

(I removed the AAT tables using ttx, to avoid modifying the font in other ways as much as possible)

[astiob]

Thanks!

Wow.

Ideally, I guess we’d like libass to mimic Windows most of the time but also to continue supporting Mac-only fonts (such as Zapfino IIRC). I suppose we could do something like: expose morx iff GSUB doesn’t exist? Would that make sense? If we take that route, are there other tables we may want to exclude in this manner, and should we check them together as all-or-nothing or each individually? Or would this be too fragile anyway?

That said, @TS40, your samples seem to assume libass-specific (VSFilter-incompatible) functionality anyway, and in that case Windows compatibility isn’t exactly a concern in the first place and it’s all the more tempting to stick to HarfBuzz’s typical behaviour. Indeed, where Windows/VSFilter compatibility is a concern, #136 actually tells us to disable ligatures entirely, except when #237 re-enables them.

[ghost]

@astiob Thank you for looking into this! I think, we can close this then.

@khaledhosny Thank you very much for the detailed analysis, that makes it very clear! I think LuaLaTeX uses HarfBuzz, too. So I tried the following:

\documentclass[a4paper]{article}
\usepackage{fontspec}
\setmainfont{LOV.Gilgengart.otf}
\begin{document}
test
\end{document}

and it looks the same as in Windows font preview. But probably some preprocessing is happening there.

[rcombs]

This might be a good case for differentiating between memory fonts and provider fonts (i.e. maybe only provider fonts on non-windows should get the non-vsfilter behavior).

[astiob]

That is also a good point!

[khaledhosny]

@astiob Thank you for looking into this! I think, we can close this then.

@khaledhosny Thank you very much for the detailed analysis, that makes it very clear! I think LuaLaTeX uses HarfBuzz, too.

It has it but it is not used by default, better try XeTeX or pass HarfBuzz option to \setmainfont.

[khaledhosny]

I suppose we could do something like: expose morx iff GSUB doesn’t exist? Would that make sense? If we take that route, are there other tables we may want to exclude in this manner, and should we check them together as all-or-nothing or each individually? Or would this be too fragile anyway?

The other main AAT table is kern, but it gets complicated because there are too versions of kern table, one is dump that Uniscribe still uses (or GDI or whatever, probably when OpenType is not used) and a smarter one that Apple uses, but then some Apple fonts mix and match AAT glyph substitution with OpenType glyph positioning and vice versa. HarfBuzz has some complicated logic to decide which table to apply, and it went through a lot of iterations.

[khaledhosny]

I forgot kerx as well (which is an extended version of the smart version of kern table).

[ghost]

@khaledhosny You are right, with XeTeX I get the behaviour that I also see with libass.

@astiob Probably the best would be to compare if there exists AAT and GSUB and if so if they differ and then print a warning. Unfortunately I'm not a C programmer so I can't really commit here. But now that I know the problem I'm ok with the FontForge font conversion workaround.

[astiob]

I assume that if it were simple to compare them, there wouldn’t be two of them in the first place… They’re probably completely different inside.

[TheOneric]

#803 reported compat issues due to morx even when there’s no GSUB alternative.

Ideally, I guess we’d like libass to mimic Windows most of the time but also to continue supporting Mac-only fonts (such as Zapfino IIRC). I suppose we could do something like: expose morx iff GSUB doesn’t exist?

This then won’t work for the sample from #803. In the particular case of #803 one might argue the libass result with morx is obviously the intended one (and it probably is), but it’s not necessarily obviously broken like the nonsense ligatures from #136 and some sub authors just inserting some negative spacing to workaround VSF results doesn’ŧ seem to far fetched. (At the same time, if we start to ignore morx this usggests we might break some libass-targeting releases)