typst
typst copied to clipboard
typst
Typst doesn't respect fontconfig `<rejectfont>` setting
Description
Currently, I only understand three languages. Till I learn the rest of the world languages, I disable the ones I don't need using fontconfig's:
<rejectfont><glob>/usr/share/fonts/TTF/*Thai*</glob></rejectfont>
> fc-list | rg 'Devanagari|Hebrew|JP|KR|Tamil|Thai' # no output
> typst fonts |rg 'Devanagari|Hebrew|JP|KR|Tamil|Thai'
Arial Hebrew
Arial Hebrew Desk Interface
Arial Hebrew Scholar
Corsiva Hebrew
Devanagari MT
Devanagari Sangam MN
IBM Plex Sans Devanagari
IBM Plex Sans Devanagari Medm
IBM Plex Sans Devanagari SmBld
IBM Plex Sans Devanagari Text
IBM Plex Sans Hebrew
IBM Plex Sans Hebrew Medm
IBM Plex Sans Hebrew SmBld
IBM Plex Sans Hebrew Text
IBM Plex Sans JP
IBM Plex Sans JP Medm
IBM Plex Sans JP Smbld
IBM Plex Sans JP Text
IBM Plex Sans KR
IBM Plex Sans KR Medm
IBM Plex Sans KR SmBld
IBM Plex Sans KR Text
IBM Plex Sans Thai
IBM Plex Sans Thai Looped
IBM Plex Sans Thai Looped Medm
IBM Plex Sans Thai Looped SmBld
IBM Plex Sans Thai Looped Text
IBM Plex Sans Thai Medm
IBM Plex Sans Thai SmBld
IBM Plex Sans Thai Text
ITF Devanagari
ITF Devanagari Marathi
Kohinoor Devanagari
Noto Rashi Hebrew
Noto Sans Devanagari
Noto Sans Devanagari UI
Noto Sans Hebrew
Noto Sans Tamil
Noto Sans Tamil Supplement
Noto Sans Tamil UI
Noto Sans Thai
Noto Sans Thai Looped
Noto Serif Devanagari
Noto Serif Hebrew
Noto Serif Tamil
Noto Serif Thai
Shree Devanagari 714
Tamil MN
Tamil Sangam MN
> typst -V
typst 0.11.1 (50115102)
Reproduction URL
No response
Operating system
Linux
Typst version
- [ ] I am using the latest version of Typst
We are using fontdb to find fonts, and it seems like it has at least minimal support for fontconfig, although I'm not sure whether it implements the feature you want. I'm not sure if there is a reason why we don't use that feature.
I'm not sure if there is a reason why we don't use that feature.
We do:
https://github.com/typst/typst/blob/b5ef9244eb2b546b47530d40860e9e04256aafb2/crates/typst-kit/Cargo.toml#L34-L37
We have default-features = false at the workspace level because typst as a library, without typst-cli, also depends on fontdb for SVG support, but not on any system stuff.
Thanks for the quick responses. Should I report it upstream or will you take care of it?
Feel free to report it upstream, though if you know a bit of Rust ideally check first that it indeed happens with just a bare fontdb as well.
Okay, I created a small code snippet using fontdb 0.21.0 and only enabled fontconfig features, and it's evident that this is a fontdb issue.
fontdb appears to have provided very rudimentary fontconfig support, possibly merely to load fonts from custom folders, so I'm not expecting a speedy fix. Would it be possible for typst to use fontconfig directly rather than relying on fontdb?
Would it be possible for typst to use fontconfig directly rather than relying on fontdb?
I'm not sure whether it would be desirable to go our own ways as opposed to improving fontdb, as it is starting to become a standard in the Rust ecosystem (e.g. also used in cosmic-text, which is fairly widely used by other projects).
But independently of that, we don't currently have the development capacity to make larger changes here.
fontdb uses the fontconfig-parser crate so there's not likely any benefit to parsing the config files manually (or directly using the fontconfig-parser crate). Before any major surgery I would throw your test case in the fontconfig-parser tests directory along with the expected output, run the tests and see if they pass.
Do we need a tag UpstreamToUpstream or what? 😀
Assuming I have done it correctly, I put my config in test-conf folder and cargo test shows all OK. Then, I used the examples/list_all_fonts.rs and it listed all the fonts ignoring the </rejectfont> element. If this is the correct way to test, should this be reported to fontconfig-parser then?
Worth noting that fontdb is intentionally seeking to keep fontconfig support very minimal as they want to avoid depending on the full thing (which would lead to a C dependency). But it could be worth it to open an issue upstream either way to see if their implementation, even if minimal, could be adapted to consider this setting.
Apologies for the wild goose chase — those tests don't actually test the font resolution.
Upon further inspection the fontconfig-parser example does what fontdb does and simply gets a list of directories from fontconfig and parses all the files found within. rejectfont entries are parsed without any C dependencies but it's up to the consumer (e.g. fontdb) to actually use them.
Spitballing here — to get the filtering working the rejectfont entries should be abstracted in a platform independent way as fontdb only uses fontconfig on non-Android Linux, then passed in to load_fonts_dir_impl.
Not much for Typst to do IMO.
typst should switch to fontconfing on Linux. That's the only option. fontdb isn't designed to replace system API.
Or simply use https://github.com/servo/font-kit
I tried font-kit and indeed it doesn't list the blacklisted fonts. Thanks for the advice
A semi-related note on this topic: another one of the many abilities of fontconfig is to alias font family names. For example, the Culmus font set (a set of open source fonts for Hebrew, often used for typesetting) installs a fontconfig configuration file that aliases its' family names to the family names of the equivalent proprietary Microsoft and Apple fonts (e.g David is registered as an alias of David CLM). That is quite convenient as it allows using the same family names in Windows and Linux. Some is done via an <alias> tag, and some is done through a <match> tag that tests the family name.
If such a thing is ever to be supported in Typst, this kind of information will be lost with any abstraction crate that just provides a stream of font file names (be it fontdb or font-kit). I think it would need direct integration with the fontconfig parser, and a selective one at that - since quite a lot of the match rules that could be specified in the system's fontconfig configuration might not make sense for Typst.
I took a pass at modifying typst-kit to make use of the "real" fontconfig library - it is not hard to integrate, and it both resolves this issue and is a good start at being able to get more specific information from fontconfig into the font book later on.
However - there may be a significant distribution / release problem with it, that I feel needs some maintainer feedback prior to spending more time on a PR.
It is fairly well known that applications should dynamically link to the system libfontconfig, and should not bundle or statically link to it (unless built in-tandem as part of a distribution). That is due to lack of any backward or forward compatibility guarantees between the library and the font configuration file format itself, and there were breaks in the past (e.g. https://bugs.freedesktop.org/show_bug.cgi?id=105818#c4). The yeslogic-fontconfig-sys crate makes doing it easy, even allowing dynamic loading of the fontconfig library at runtime.
The difficulty is the prebuilt typst-cli binaries that are released with Typst - which are statically linked with musl. musl static builds can't use dlopen, and even if they could - loading a likely glibc-linked library from the system into a musl-based binary would cause massive havoc. In such binaries initializing fontconfig just gracefully fails. It probably has some value to just have it as an optional Cargo feature for external integrators and people and Linux distributions building their own CLI; or it could cause confusion, I don't know.
The difficulty is the prebuilt
typst-clibinaries that are released with Typst - which are statically linked withmusl.muslstatic builds can't usedlopen, and even if they could - loading a likely glibc-linked library from the system into a musl-based binary would cause massive havoc. In such binaries initializing fontconfig just gracefully fails. It probably has some value to just have it as an optional Cargo feature for external integrators and people and Linux distributions building their own CLI; or it could cause confusion, I don't know.
as you are already linking statically, why not also statically link fontconfig ?
fontconfigs standard directories are mostly XDG standardized, very-well-known, or given in the config files themselves.
challenges that arise for static linking of fontconfig are also static linking of at least freetype2 and expat as requisites tho.
as you are already linking statically, why not also statically link fontconfig ?
fontconfigs standard directories are mostly XDG standardized, very-well-known, or given in the config files themselves.
The issue is not so much that one CAN'T statically link fontconfig (it is involved and poorly supported by the current crate ecosystem, but should be possible) - it is that one SHOULDN'T.
The problem is not where to find the font configuration XML files - it is that they (or at least the base ones in conf.avail and conf.default) are supposed to be shipped together with the fontconfig library and my understanding of the position of its' maintainers is that they need to match versions. You can't use an old library with newer files (that has caused breakage before), and you also probably can't use a new library with older files (I'm not aware of actual issues around that, but without a guarantee that it will work it is hard to rely on that).
This is a relevant problem, as my understanding the pre-built CLI is supposed to be something self-contained and portable - so unless the CLI will also bundle a full set of configuration files (which would be quite silly, I suppose), it simply can't both use the real fontconfig library and and be statically linked at the same time.
Therefore I am looking for a maintainer indication of their intentions around this - i.e. would having this as an optional feature that downstreams (like tinymist and Linux distributions building their own packages of the CLI) can opt-in to but the official binaries won't is acceptable or not.
@laurmaedje may I ask for your opinion on the above? Thank you!
I am not per se against using system interfaces where appropriate (e.g. also using DirectWrite and Core Text on Windows/macOS), but I am concerned both with
- Having heavily diverging behaviour between distro packaged CLIs and ones downloaded from our releases.
- Portability of the release binaries (they should work on everything from normal desktop OSs, to servers, and embedded devices).
On a server it's typically more appropriate to provide fonts via font paths anyway, so opening the fontconfig library dynamically and failing gracefully when it's not present in the distro would be ideal. But the fact that this doesn't work with musl is a big problem :(
On a server it's typically more appropriate to provide fonts via font paths anyway, so opening the fontconfig library dynamically and failing gracefully when it's not present in the distro would be ideal. But the fact that this doesn't work with musl is a big problem :(
In this case, the only path I see available right now is to shell out to the fc-list binary (if found in the PATH), and query the fontconfig cache indirectly through it - unsavory as such approaches are.
If not that, it will be just sit and wait until something in the ecosystem shifts; which who knows when or if that will happen. So far efforts to create a pure Rust drop-in alternative to fontconfig have not gone far (and who can blame them). Not even Fontations seems to look to replace fontconfig itself ATM - only to integrate into it...