ducalex
Changing font format to LVGL embedded format and adding a python tool to create fonts
- Adding diacritics in translations.h
- TODO : Creating a python tool for font generation
- TODO : Updating Fonts
(Actually I innitially wanted to generate fonts with the .exe and then create the .py for later use but If think I'm gonna start with the .py)
So I have created a very crude python font converter Todo:
- improve quality (~the same as the exe if possible)
- make it more user friendly
- Make a GUI
I'm impressed that you've already made a python tool, this is nice!
Now that retro-go will ship with a generator, I think we should add the .ttf (or whatever) fonts to the repo to preserve as reference if someone wants to regenerate the .c fonts with more glyphs or sizes.
Actually I cheated a bit, I used ChatGPT to help me with the .ttf font extraction ;) But understanding the custom encoding for the output took me a few hours 😅 I documented all my research directly at the beginning of the python converter for future dev.
Right now the tool does not make results as good as the .exe I still have to do some adjustments.
I agree to add the .tff in the repo !
I created a UI with a font renderer ! To make it easier to change font settings
Oh wow, having a preview is a game changer! I remember having to flash retro-go 50+ times back when I was trying to find the perfect parameters for the .exe generator...
Yeah absolutely I've been through this struggle too with Open Sans but now it's gonna be way faster ^^ I'm currently trying to add a way to directly "edit" the c font like Paint software directly on the canvas that would be the cherry on the cake for the tool !
I was looking on the web for inspirations from other tools and I came across LVGL font converter https://lvgl.io/tools/fontconverter This tool uses a special type to store the "header" for each character and precomputes the index of the first byte of each character. I feel like this approach is better than Retro-Go's one in terms of performance. Only issue is that it uses a bit more memory to store the indexes (probably around 16bit(2 Bytes) more per character) In the end we should loose around 161*2 = 322 Bytes / 2.5Kbit per font or 1.6KBytes / 12.8Kbit for 5 fonts.
So I was wondering if it was worth the try since we are making our own tool ?
/*-----------------
* BITMAPS
*----------------*/
/*Store the image of the glyphs*/
static LV_ATTRIBUTE_LARGE_CONST const uint8_t glyph_bitmap[] = {
/* U+0020 " " */
0x0,
/* U+0021 "!" */
0xff, 0xcf,
/* U+0022 "\"" */
0xff, 0xf0,
/* U+0023 "#" */
0x16, 0x14, 0x7f, 0x24, 0x24, 0xfe, 0x28, 0x68,
/* U+0024 "$" */
0x10, 0xf9, 0x43, 0x87, 0xc3, 0xc5, 0xbe, 0x10,
/* U+0025 "%" */
0x71, 0x1b, 0x23, 0x68, 0x3a, 0x0, 0x5c, 0x16,
0xc4, 0xd8, 0x8e,
/* U+0026 "&" */
0x3c, 0x30, 0x1c, 0x6, 0xf, 0xb6, 0x7b, 0x18,
0xfe,
/* U+0027 "'" */
0xfc,
/* U+0028 "(" */
0x6f, 0x6d, 0xb3, 0x60,
/* U+0029 ")" */
0xd9, 0xb6, 0xde, 0xc0,
};
/*---------------------
* GLYPH DESCRIPTION
*--------------------*/
static const lv_font_fmt_txt_glyph_dsc_t glyph_dsc[] = {
{.bitmap_index = 0, .adv_w = 0, .box_w = 0, .box_h = 0, .ofs_x = 0, .ofs_y = 0} /* id = 0 reserved */,
{.bitmap_index = 0, .adv_w = 61, .box_w = 1, .box_h = 1, .ofs_x = 0, .ofs_y = 0},
{.bitmap_index = 1, .adv_w = 80, .box_w = 2, .box_h = 8, .ofs_x = 2, .ofs_y = 0},
{.bitmap_index = 3, .adv_w = 92, .box_w = 4, .box_h = 3, .ofs_x = 1, .ofs_y = 5},
{.bitmap_index = 5, .adv_w = 147, .box_w = 8, .box_h = 8, .ofs_x = 1, .ofs_y = 0},
{.bitmap_index = 13, .adv_w = 122, .box_w = 7, .box_h = 9, .ofs_x = 0, .ofs_y = -1},
{.bitmap_index = 21, .adv_w = 176, .box_w = 11, .box_h = 8, .ofs_x = 1, .ofs_y = 0},
{.bitmap_index = 32, .adv_w = 153, .box_w = 9, .box_h = 8, .ofs_x = 1, .ofs_y = 0},
{.bitmap_index = 41, .adv_w = 54, .box_w = 2, .box_h = 3, .ofs_x = 1, .ofs_y = 5},
{.bitmap_index = 42, .adv_w = 80, .box_w = 3, .box_h = 9, .ofs_x = 1, .ofs_y = -1},
{.bitmap_index = 46, .adv_w = 80, .box_w = 3, .box_h = 9, .ofs_x = 1, .ofs_y = -1},
{.bitmap_index = 50, .adv_w = 92, .box_w = 5, .box_h = 5, .ofs_x = 0, .ofs_y = 3},
{.bitmap_index = 54, .adv_w = 147, .box_w = 7, .box_h = 6, .ofs_x = 1, .ofs_y = 0},
{.bitmap_index = 60, .adv_w = 67, .box_w = 3, .box_h = 3, .ofs_x = 0, .ofs_y = -2},
{.bitmap_index = 62, .adv_w = 73, .box_w = 3, .box_h = 1, .ofs_x = 1, .ofs_y = 3},
{.bitmap_index = 63, .adv_w = 67, .box_w = 2, .box_h = 2, .ofs_x = 1, .ofs_y = 0}
};
I feel like this approach is better than Retro-Go's one in terms of performance. Only issue is that it uses a bit more memory to store the indexes (probably around 16bit(2 Bytes) more per character) In the end we should loose around 161*2 = 322 Bytes / 2.5Kbit per font or 1.6KBytes / 12.8Kbit for 5 fonts.
So I was wondering if it was worth the try since we are making our own tool ?
Yes, if you feel there is a better way then you can rewrite our font format and renderer to something completely different. And if you convert the basic font to your new format then we could eventually ditch the two other rendering modes as well.
It would be nice to have a single format for all fonts at last, that is fully documented and with a generator.
The size of the font in flash doesn't bother me, as long as your new renderer doesn't use more stack space and isn't slower than the current one, I will accept it happily :) If anything a 16bit index seems like it would be a win, we could add more like languages like cyrillic glyphs or even add some custom icons to replace things like the wifi/battery icons.
If you do decide to write a new renderer, I'd request you update your PR description to match this new goal (for clarity).
I'm going to tag a new retro-go release before merging this PR. I want to be sure that this PR is merged in a known stable base because every time I touch the rendering code it introduces subtle bugs and this time won't be any different I'm sure...
What's the status on your side? Do you feel this PR is ready for review from my side?
I'm going to tag a new retro-go release before merging this PR. I want to be sure that this PR is merged in a known stable base because every time I touch the rendering code it introduces subtle bugs and this time won't be any different I'm sure...
What's the status on your side? Do you feel this PR is ready for review from my side?
On my side :
- the renderer code is done (I might do some adjustment on this line: yOffset = glyph_dsc[c].ofs_y-2)
- the C font editor python program is 99% working but in the future i'd like to improve it to get more personnalization options (+ icons creation)
- the font converter is working and produces decent results but not 100% pixel perfect. I have to find the right settings for the advance_width but it's just a matter of time.
Lastly I have to write some documentation on the format
I updated the font editor so we can now select a single character on the preview and edit it, here is a screenshot:
I now have to add more personalizations options...
I'd like to cleanup some things (missing boundary checks, formatting, fix the conflicts) to get ready to merge.
I think I'm allowed to commit to your branch directly. Is it okay with you if I do?
Of course you can, no problem ;) Also, a little tip: there not the same count of fonts in this branch, so the SD card settings of the regular branch for the font must be cleared (or boot without an sd card)
Now that I've been working on this, considering the new format implemented here isn't really compatible with LGVL I don't think it's worth changing the format that we use until we implement enough of LVGL rendering to be able to take their .c font files directly.
However your tooling is clearly a good thing because right now retro-go is depending on some random .exe to generate new fonts. So I'd like to tweak the tools to output our current format instead (fairly minor change to save_file).
So this PR would become only about adding the font tools and french translation diacritics.
What do you think?
Now that I've been working on this, considering the new format implemented here isn't really compatible with LGVL I don't think it's worth changing the format that we use until we implement enough of LVGL rendering to be able to take their .c font files directly.
However your tooling is clearly a good thing because right now retro-go is depending on some random .exe to generate new fonts. So I'd like to tweak the tools to output our current format instead (fairly minor change to
save_file).So this PR would become only about adding the font tools and french translation diacritics.
What do you think?
I apologize for the late answer, Yeah you are right I stripped down a lot of LVGL specifics so it wasn't really the LVGL format anymore. I also feel like it would be better to handle the original LVGL font directly! Lastly you can modify the font tool to your will (I saw you already changed it a bit)!
Thanks, so I'll finalize removing support for your hybrid format and fix the conflicts. I'm not sure how to do that just yet, I have to somehow update your branch to be closer to current dev. I'll probably make a git mess... But anyhow, in the end it will all be squashed and my mess will be gone and you'll retain authorship of your contribution :)
Now that retro-go supports utf-8 I'm eager to merge your diacritics and the font converter is an essential addition in my opinion!
I have some more work I'd like to do, like improve the editor and merge it with the converter because it makes more sense. But right now the converter works well, I used to update all current fonts! So I think I'll merge this PR and do further work later in my own branch.
Thank you very much for your contribution to translations and font tooling!
Oh, and I'll eventually re-add the OpenSans font. I just forgot to regenerate it when I deleted the lvgl version...
I have some more work I'd like to do, like improve the editor and merge it with the converter because it makes more sense. But right now the converter works well, I used to update all current fonts! So I think I'll merge this PR and do further work later in my own branch.
Thank you very much for your contribution to translations and font tooling!
Thank you too for taking the time to review and adapt my original code !
Also I saw on one commit you talked about the max_height being higher because of diacritics. I don't know if this is still a thing but what I personally did to avoid this issue was just to convert the font without including the capital letters with diacritics since they aren't used in retro-go currently and are easy to avoid. Altough we might need them in the future...
I've spent a lot of time tweaking the height and trying to find ways to reduce it to at least match the older font. But in the end I kind of gave up on the code being clever, because I couldn't get consistent results across all fonts. The result is DejaVu is 2px taller and VeraBold 1px taller.
For example shifting '¯ÀÁÂÃÄÅÈÉÊËÌÍÎÏÑÒÓÔÕÖÙÚÛÜÝå' down and 'Çç¸' up reduces max_height by 1px in DejaVu and 2px in Arial, but 0px in VeraBold or OpenSans.
I was hoping to eventually add basic editing tools to font_converter to allow us to shift individual glyphs manually instead. But in retrospect maybe I should still have generated DejaVu with the shifting until the fancy editor arrives. Because it would have saved 1px which is about 1 more line of text that could fit on screen.