verovio icon indicating copy to clipboard operation
verovio copied to clipboard

Refactor font generation

Open ahankinson opened this issue 2 years ago • 2 comments

This PR refactors the font generation process. Several Python scripts are combined into a new utility, generate.py which currently handles the following tasks:

  • generates the smufl.h file containing the supported glyphs
  • extracts the supported glyphs from an SVG font
  • converts a font to a woff2 font and, optionally, wraps it in a base64-encoded CSS @font-face definition.
  • converts a font to a SVG font

The last two require a fontforge installation to run.

lxml has been deprecated since the attribute ordering problem was fixed in Python 3.8.

svgpathtools are still required, but since it's so heavy (requiring numpy and scipy, which requires LAPACK/BLAS and a fortran compiler) suggestions for alternate solutions are welcome!

The supported.xsl file has been changed to supported.xml, and the XSLT functions removed. It was not being used as an XSLT anyway, so it was decided to change this.

From conversations with @lpugin the generate.py script does not currently handle text fonts, unlike previous versions. This can be changed if desired.


generate.py uses the built-in Python argparse module, which provides some additional niceties for the command-line utility.

python3 generate.py --help should provide help about the script itself. The individual functions are implemented as sub-commands, so python3 generate.py woff2 --help will provide help specific to the woff2 sub-command.


How to upgrade:

The structure of the fonts directory has changed significantly. Each font now has its own directory, and the generate.py script is built to work against this. If you add new fonts, please follow the Fontname/Fontname.[ttf|otf] standard. You can specify a custom directory for this with the --source parameter on the extract command, but then the structure should still be /your/custom/fonts/Fontname/Fontname.[ttf|otf] if the command is --source /your/custom/fonts/

Likewise, the SMuFL metadata for each font has been moved to the font directory. It should follow the fontname_metadata.json standard. (fontname should be lower-case here, e.g., bravura_metadata.json)

Where previously you might have called: python3 generate-smufl-code.py, you can now call python3 generate.py smufl.

Where previously you might have called: python3 extract-bounding-boxes.py Bravura you can now call python3 generate.py extract Bravura

Where you might have called ./generate_woff2.sh you can now call python3 generate.py woff2 Bravura Bravura/Bravura.otf --wrap

The ./generate_all.sh script has been updated with the new functions.


All of the data has been re-generated with this script, as a test to ensure that it produces no significant differences. As a reviewer, please check that you do not see any signficant differences. The most changes seem to be an additional space added between the last attribute and the self-closing end-tag, as a result of moving to the built-in ElementTree writer.

ahankinson avatar Sep 21 '22 14:09 ahankinson

Results for the run 3098529098

1 change(s) detected in 1 test category(ies) ****** cross-staff: 1 3028 dev-dir gh-pages-dir pr-653c987da62dab9f96a6b4c1f035b96c528b232f temp-dir wg0.conf

github-actions[bot] avatar Sep 21 '22 14:09 github-actions[bot]

NB: After review with @lpugin a few changes to WOFF and SVG generation are incoming.

ahankinson avatar Sep 22 '22 08:09 ahankinson

@rettinghaus it's probably a good idea to have a look at this, as we think it might have an impact on your workflows!

ahankinson avatar Sep 26 '22 07:09 ahankinson