Make OTF latest version big disaster!

[astype]

[Last working Version / Windows 10 Pro 64bit / makeotf.py v2.9.0 (hotconv 1.0117; makeotfexe 2.5.65602) June 1 2021]

For my projects I use splitted source files stripped together with "include"

sample:

include(../0A_family_metrics.txt);

Panose 2 11 2 3 3 0 3 2 5 4;
       XHeight 520; 
       CapHeight 680;
       FSType 4;
       WeightClass   	250;
       WidthClass    	5; 
       Vendor "ASSA";    

} OS/2;


include(../0B_family_base_table.txt);
include(../0C_family_name_table.txt);

I get these errors:

  ERROR: [line     5 char   1] extraneous input 'Panose' expecting {<EOF>, 'anon', 'anonymous', 'include', 'feature', 'table', 'languagesystem', 'lookup', 'anchorDef', 'valueRecordDef', 'markClass', GCLASS}
makeotfexe in feature file '../0A_family_metrics.txt' (included from _head.txt):
  ERROR: [line    42 char   4] extraneous input '<EOF>' expecting {'include', 'FSType', 'fsType', 'LowerOpSize', 'UpperOpSize', 'Panose', 'TypoAscender', 'TypoDescender', 'TypoLineGap', 'winAscent', 'winDescent', 'XHeight', 'CapHeight', 'WeightClass', 'WidthClass', 'Vendor', 'UnicodeRange', 'CodePageRange', 'FamilyClass', '}'}
makeotfexe in feature file '../0C_family_name_table.txt' (included from _head.txt):
  ERROR: [line     2 char  12] mismatched input 'include' expecting {QUOTE, NUMEXT, NUMOCT, NUM}
  ERROR: [line     3 char  13] mismatched input 'include' expecting {QUOTE, NUMEXT, NUMOCT, NUM}
  ERROR: [line     5 char  12] mismatched input 'include' expecting {QUOTE, NUMEXT, NUMOCT, NUM}
  ERROR: [line     6 char  14] mismatched input 'include' expecting {QUOTE, NUMEXT, NUMOCT, NUM}
  ERROR: [line    11 char  12] mismatched input 'include' expecting {QUOTE, NUMEXT, NUMOCT, NUM}
  ERROR: [line    12 char  14] mismatched input 'include' expecting {QUOTE, NUMEXT, NUMOCT, NUM}
  ERROR: [line    14 char  13] mismatched input 'include' expecting {QUOTE, NUMEXT, NUMOCT, NUM}
  ERROR: [line    15 char  15] mismatched input 'include' expecting {QUOTE, NUMEXT, NUMOCT, NUM}
  ERROR: [line    23 char  12] mismatched input 'include' expecting {QUOTE, NUMEXT, NUMOCT, NUM}
  ERROR: [line    24 char  14] mismatched input 'include' expecting {QUOTE, NUMEXT, NUMOCT, NUM}
  ERROR: [line    32 char  12] mismatched input 'include' expecting {QUOTE, NUMEXT, NUMOCT, NUM}
  ERROR: [line    33 char  14] mismatched input 'include' expecting {QUOTE, NUMEXT, NUMOCT, NUM}
makeotfexe in feature file '../../copyright_pc.txt' (included from ../0C_family_name_table.txt):
  ERROR: [line     1 char   1] mismatched input '"' expecting {<EOF>, 'include', 'nameid'}
makeotfexe in feature file '../../copyright_mac.txt' (included from ../0C_family_name_table.txt):
  ERROR: [line     1 char   1] mismatched input '"' expecting {<EOF>, 'include', 'nameid'}
makeotfexe in feature file '../../trademark.txt' (included from ../0C_family_name_table.txt):
  ERROR: [line     1 char   1] mismatched input '"' expecting {<EOF>, 'include', 'nameid'}
makeotfexe in feature file '../../trademark.txt' (included from ../0C_family_name_table.txt):
  ERROR: [line     1 char   1] mismatched input '"' expecting {<EOF>, 'include', 'nameid'}

I'm within a release cycle - any options to role back to a previews working version with pip? Thank you.

[punchcutter]

With pip you can do pip install afdko==3.6.2 to get the last version.

[astype]

I managed to use the recovery point of my windows system - so I'm now on makeotf.py v2.9.0 (hotconv 1.0117; makeotfexe 2.5.65602) from June 1 2021 - with is more up to date I think.

But I am a bit shocked how such serious bug can be implemented. Something very fundamental has been changed. What is this?

[frankrolf]

The OS/2 table is opened within the 0A_family_metrics.txt file, presumably? The include system still works, but I am not sure it was expected to work across a feature “fence”.

Could you share more of the feature code, or the actual feature files? Thanks!

[astype]

It worked before the latest updates very well. I can send you a project zip file? Can you please send me an email address? I don't like to share it in public. Thank you.

[frankrolf]

Please send to ~~redacted~~ if you agree your project will be shared (within our team) for debugging purposes.

[astype]

Thank you. The ZIP is on its way.

[frankrolf]

Received, thanks!

[frankrolf]

I have analyzed your files, and will return them to you shortly. Here’s a summary:

• indeed, the feature file parser has changed. Significant effort has been made to replace the the aging, home-grown, parser with antlr4.
• this is a major change, and it may require some re-working of feature files, particularly ”creative” include statements, but this change also helps the OT feature parsing be much more predictable and stable.
• feature files need to be UTF-8 encoded. The files supplied were encoded in a mix of UTF-8 and MacRoman encodings.

We are taking reports like yours seriously, and are trying to make the change as seamless as possible – however, we cannot predict all use cases. In the meantime, we are also trying to make the parser better, by suggesting improvements like this: https://github.com/antlr/antlr4/issues/3280

[astype]

Thank you Frank, hope my code will work in future without big changes. :-)