Please fix non latin support

[yarda]

Describe the bug Use e.g. the following text "Šutr". It previews OK on android, but when the text is sent to the badge everything after the "Š" is garbled. It seems there is some overflow, because the "utr" is wrongly vertically aligned and unreadable.

To Reproduce Use e.g. the text "Šutr".

Expected behavior Text "Šutr" is correctly rendered on both android preview and on the badge.

Logs Not applicable.

Screenshots

Smartphone Info:

Device	Xperia XZ2C
Android Version	9

Additional context

Would you like to work on the issue? I can try to fix it.

[icedwater]

Hey @yarda! Thanks for raising this issue. I was testing some German characters and found similar alignment problems. Have you managed to find some clues on how to solve this? I hope I can find some time to look into the code as well.

[jaivsh]

Hi, I can try to fix this

[icedwater]

I've added some characters to the font on my end but I've only tested the USB version from upstream, not the Android app yet. I think that's the way forward, adding new characters - it may be intractable in the long run but as long as we're not planning to support ALL unicode characters I think ad-hoc additions might suffice.

Maybe a script to convert unicode glyphs into 8x11 bitmaps?

[fordfisher]

I have same issue with "ö" - looks like UTF issue. Any progress @jaivsh ?

[yarda]

Well, it seems the app was ported to Flutter and during the port the UTF-8 support was dropped. Initially, it worked the way, that for characters it didn't know (usually UTF-8 beyond the ASCII table), it rendered the UTF-8 characters to the bitmap from the TTF which was used as the source for the LED display. The code was buggy and sometimes the scaled down characters looked weird, but it works somehow - see the screenshots in my first comment. I.e. the "Š" looked weird in the app, but it was there. Now, this functionality was dropped and it uses only ASCII letters from the pre-generated table https://github.com/fossasia/badgemagic-app/blob/76eec14772b6d4a62372b4dd851357a1f50294dd/lib/bademagic_module/utils/data_to_bytearray_converter.dart#L13-L107 The characters in the table are polished and looks good, but if you use any character it doesn't know, it's simply ignored now.

Somebody could try to either:

• implement the TTF bitmaps rendering/scaling in the Dart, similarly to the previous code, but the result would probably also look ugly, this would support all UTF-8 characters including the CJK/arabic, etc. or
• at least support the most used latin characters and manually add their bitmaps to the conversion table. Maybe there is a good usable and open 8x11 bitmap font somewhere on the internet that could be reused and accelerate this task? Or maybe with some effort usable results could be also obtained from the tuned TTF renderer to pre-generate the 8x11 bitmaps?

[yarda]

If I am counting correctly, for latin it's 61 missing characters, most of them are very similar to what's already there. It shouldn't be that much work for manual/semi-manual conversion. I will try to prepare something. I tried to use AI for this task, but the result was much worse than the output from the TTF 8x11 generator.

[yarda]

Well, this went faster than I initially thought :). Feel free to improve the characters (I really didn't spend much time on it :)), I also hacked very crude editor that can simplify edit or addition of future characters: https://github.com/yarda/BMPFontDesigner