[VARC] Variable Composites table

[behdad]

Based on https://github.com/harfbuzz/boring-expansion-spec/issues/103#issuecomment-1856325577

[behdad]

This is mostly steady now. I'll go ahead and write a spec for it.

What is left to do is:

• [x] Rip out the glyf1 VarComposite glyphs,
• [x] Hook VARC up in ttGlyphSet so we can draw.

[behdad]

I should also somehow figure out how to do the VARC glyph loading lazy.

[behdad]

TODO:

• [x] Make VARC table lazy-load glyphs.
• [x] Subsetter
• [x] Instancer
• [x] scaleUpem
• [x] tests

[behdad]

I can use some directions here. The multiVarStore and varStore share some similar code. Should I try to subclass them from a common class?

It will be more pronounced when adding subsetting to multiVarStore.

[behdad]

I can use some directions here. The multiVarStore and varStore share some similar code. Should I try to subclass them from a common class?

It will be more pronounced when adding subsetting to multiVarStore.

I managed to share some code by monkey-patching for now.

[behdad]

This is ready for review. Thanks in advance @anthrotype @justvanrossum

[behdad]

This is ready for review. Thanks in advance @anthrotype @justvanrossum

It's terse but here's something of a spec for this: https://github.com/harfbuzz/boring-expansion-spec/blob/main/varc.md

[behdad]

I probably should move some of the new code from otTables to a side module.

[anthrotype]

The multiVarStore and varStore share some similar code. Should I try to subclass them from a common class? It will be more pronounced when adding subsetting to multiVarStore. I managed to share some code by monkey-patching for now.

It may be cleaner to subclass from a common base class, or try to factor out the shared code into separate objects or helpers and compose them accordingly to better clarify what is shared/different between the two. It's not immediatly obvious what the effect of the line import fontTools.varLib.varStore # For monkey-patching at the top of multiVarStore.py is.

[anthrotype]

I'm looking at the ttx dump of VARC table as implemented in this PR and I wonder if we could improve it a bit, make more human friendly. For example, the way TupleValues are written out as concatenation of the tuple deltas for each region: in the old (item) VarStore, each row of deltas would contain the same number of deltas, one per region, and would look more like an actual table; whereas each multi-item's length now is variable and equals number of regions x number of values per tuple; sometimes it can get quite long and wrap to the new line.. I wonder if we could split this long concatenated list of numbers into the tuples that comprise it, one per line, so each line doesn't get exceedingly long and the grouping is explicit. I haven't fully thought this through..

[anthrotype]

the VarComponent element of the ttx dump can also get quite long and wide with all the optional attributes , I wonder if it would look better to have separate sub-elements for each.

So in stead of this:

<VarComponent index="0" glyphName="glyph00006" flags="3967" axisIndicesIndex="8" axisValues="(3621, 836, 9093, 13959, 13795, 5800, 1540, 0, 16384)" axisValuesVarIndex="262144" transformVarIndex="262145" translateX="-456.0" translateY="558.0" rotation="-231.0205" scaleX="0.64" scaleY="0.95" tCenterX="973.0" tCenterY="10.0"/>

Something like this:

<VarComponent index="0">
  <glyphName value="glyph00006"/>
  <flags="3967"/><!-- do we actually need this? -->
  <axisIndicesIndex value="8"/>
  <axisValues value="(3621, 836, 9093, 13959, 13795, 5800, 1540, 0, 16384)"/>
  <axisValuesVarIndex value="262144"/>
  <transformVarIndex value="262145"/>
  <translateX value="-456.0"/>
  <translateY value="558.0"/>
  <rotation value="-231.0205"/>
  <scaleX value="0.64"/>
  <scaleY value="0.95"/>
  <tCenterX value="973.0"/>
  <tCenterY value="10.0"/>
</VarComponent>

[behdad]

  <flags="3967"/><!-- do we actually need this? -->

Thanks Cosimo. Yes, the flags are needed mostly to know which transform components get encoded, because they have have variations, so we cannot simply encode any non-default transform component.

[behdad]

the VarComponent element of the ttx dump can also get quite long and wide with all the optional attributes , I wonder if it would look better to have separate sub-elements for each.

So in stead of this:

<VarComponent index="0" glyphName="glyph00006" flags="3967" axisIndicesIndex="8" axisValues="(3621, 836, 9093, 13959, 13795, 5800, 1540, 0, 16384)" axisValuesVarIndex="262144" transformVarIndex="262145" translateX="-456.0" translateY="558.0" rotation="-231.0205" scaleX="0.64" scaleY="0.95" tCenterX="973.0" tCenterY="10.0"/>

Something like this:

<VarComponent index="0">
  <glyphName value="glyph00006"/>
  <flags="3967"/><!-- do we actually need this? -->
  <axisIndicesIndex value="8"/>
  <axisValues value="(3621, 836, 9093, 13959, 13795, 5800, 1540, 0, 16384)"/>
  <axisValuesVarIndex value="262144"/>
  <transformVarIndex value="262145"/>
  <translateX value="-456.0"/>
  <translateY value="558.0"/>
  <rotation value="-231.0205"/>
  <scaleX value="0.64"/>
  <scaleY value="0.95"/>
  <tCenterX value="973.0"/>
  <tCenterY value="10.0"/>
</VarComponent>

Done.

[behdad]

For example, the way TupleValues are written out as concatenation of the tuple deltas for each region: in the old (item) VarStore, each row of deltas would contain the same number of deltas, one per region, and would look more like an actual table; whereas each multi-item's length now is variable and equals number of regions x number of values per tuple; sometimes it can get quite long and wrap to the new line.. I wonder if we could split this long concatenated list of numbers into the tuples that comprise it, one per line, so each line doesn't get exceedingly long and the grouping is explicit. I haven't fully thought this through..

The VarStores are inherently not human-friendly, so I'm not sure how important this is. Currently there is no custom toXML involved, it's all automatic from:

    (       
        "MultiVarData", 
        [ 
            ("uint8", "Format", None, None, "Set to 1."), 
            ("uint16", "VarRegionCount", None, None, ""), 
            ("uint16", "VarRegionIndex", "VarRegionCount", 0, ""), 
            ("TupleList", "Item", "", 0, ""), 
        ], 
    ), 

If I scratch my head enough I can implement what you suggest...

[Lorp]

I'm a bit surprised to see axisValues as int16 rather than as float representing F2DOT14. Do we do this anywhere else in TTX?

[behdad]

I'm a bit surprised to see axisValues as int16 rather than as float representing F2DOT14. Do we do this anywhere else in TTX?

That's a good point. I'll see what I can do.

[behdad]

I'm a bit surprised to see axisValues as int16 rather than as float representing F2DOT14. Do we do this anywhere else in TTX?

So if I keep these in F2DOT14 in the Python objects, that's fine, just more work when applying variations to them.

[behdad]

I'm a bit surprised to see axisValues as int16 rather than as float representing F2DOT14. Do we do this anywhere else in TTX?

Fixed.

[Lorp]

Thanks Cosimo. Yes, the flags are needed mostly to know which transform components get encoded, because they have have variations, so we cannot simply encode any non-default transform component.

I don’t follow this. The flags indicate which transforms exist, and that is expressed in TTX by listing them. Whether transforms are variable is a single flag. Therefore, when encoding, flags can be deduced from the presence of axisValuesVarIndex, transformVarIndex and individual transforms.

[anthrotype]

yeah that was my thinking too. For all the optional values that are behind a flag, their mere presence entails the respective flag HAVE_{SOMETHING} is set. They should not be omitted from the ttx dump if their value is equal to the default, only if their optional flag is unset. I think there is still one flag 0 RESET_UNSPECIFIED_AXES that is standalone and not related to any of the optional fields; the GID_IS_24BIT can also be inferred by the size of the gid value. Since there's at least one flag that cannot be inferred we should keep the flags in the ttx dump, but perhaps we could clear the bits that can be computed (like we do for glyf components flags as well)

[behdad]

Okay. I'll take care of flags now.

[behdad]

Okay. I'll take care of flags now.

Done. I still read an optional flags tag...

[behdad]

Currently I'm outputing:

          <resetUnspecifiedAxes value="True"/>

for the lone necessary flag only if it's set. Should I remove the value attribute from it?

[anthrotype]

Done. I still read an optional flags tag...

so you read it but don't write it out any more? what's the use of that?

Currently I'm outputing: <resetUnspecifiedAxes value="True"/> for the lone necessary flag only if it's set. Should I remove the value attribute from it?

As in attribute-less element i.e. <resetUnspecifiedAxes/>? How about you write resetUnspecifiedAxes="True" instead as an actual xml attribute of the VarComponent element, or actually closer to where it's being used, an attribute of axisValues element?

[anthrotype]

note that for a similar boolean flag elsewhere we write "1" instead of "True" in the xml dump, you may want to do the same for resetUnspecifiedAxes:

https://github.com/fonttools/fonttools/blob/679a5c0c252fd5b983de0266b0a250fc7cbb49fc/Lib/fontTools/ttLib/tables/_g_l_y_f.py#L788-L789

[behdad]

Done. I still read an optional flags tag...

so you read it but don't write it out any more? what's the use of that?

Nothing honestly. I'll remove it.

Currently I'm outputing: for the lone necessary flag only if it's set. Should I remove the value attribute from it?

As in attribute-less element i.e. <resetUnspecifiedAxes/>? How about you write resetUnspecifiedAxes="True" instead as an actual xml attribute of the VarComponent element, or actually closer to where it's being used, an attribute of axisValues element?

Okay I'll add it to axisValues, and make it output 1. Default to 0? Or always output it?

[anthrotype]

I'd say default to 0 and only write if 1

[behdad]

XML output for resetUnspecifiedAxes adjusted as per feedback. Reading of flags removed.

[behdad]

HarfBuzz counterpart WIP: https://github.com/harfbuzz/harfbuzz/pull/4578

[behdad]

HarfBuzz counterpart WIP: harfbuzz/harfbuzz#4578

Now can render fonts generated with this fonttools branch.