wasm4 icon indicating copy to clipboard operation
wasm4 copied to clipboard

Non-ASCII text seems to be garbled.

Open vi opened this issue 2 years ago • 10 comments

Creating template Rust project and modifying string literal in the first text call to contain non-ASCII UTF-8 bytes makes the game output garbled text, as if it were interpreting the bytes it as CP1252 instead of UTF-8.

This contradicts with the name of function textUtf8.

Inspecting the cartridge with wasm2wat shows that the string inside is indeed UTF-8.

vi avatar Sep 03 '21 13:09 vi

Yeah, text actually only supports a limit character set. The font only has characters in the 32-255 range. I think we need to document that, and also add some bounds checking for invalid characters.

aduros avatar Sep 03 '21 21:09 aduros

How games relying on textUtf8 are expected to be localized then?

vi avatar Sep 03 '21 21:09 vi

Also shall the font at least include ↑↓←→ characters, so that games can explain which buttons need to be pressed easily?

vi avatar Sep 04 '21 13:09 vi

How games relying on textUtf8 are expected to be localized then?

The problem is that the bitmap font used by text() doesn't have non-ASCII characters, and even if it did, adding all UTF-8 characters would increase the file size and memory usage of the runtime by a lot.

I think the best solution is to include your own font sheet in the cartridge and handle the drawing using blitSub.

Also shall the font at least include ↑↓←→ characters, so that games can explain which buttons need to be pressed easily?

Sure, that sounds like a good idea. I'll commit the image used to generate the built-in runtime font. There are a few unused spaces that could be filled with useful glyphs like this.

aduros avatar Sep 04 '21 19:09 aduros

Maybe browser implementation of Wasm4 should just cheat and use browser's font in case of tricky text, rendering it not in framebuffer, but just as a separate element that is positioned approximately where proper in-game text would be?

vi avatar Sep 05 '21 03:09 vi

Also shall the font at least include ↑↓←→ characters

You could use ordinal bitmaps for this. Another option is allowing set custom font atlas I guess

MaxGraey avatar Sep 05 '21 12:09 MaxGraey

Maybe browser implementation of Wasm4 should just cheat and use browser's font in case of tricky text

This isn't ideal because a future goal of WASM-4 is to run outside of the browser. Rather than text rendering differently in different runtimes (and even different browsers!) I think it would be best for carts to bring their own font.

Also shall the font at least include ↑↓←→ characters

I just committed the font image, which is a sprite sheet of 8x8 glyphs vertically stacked, starting at ASCII code 20 (space). There are lots of empty characters starting at code 128, which would be a good place to draw in new glyphs like these. After making changes to the image use w4 png2src to update the code

Then games can pass strings like this, for example:

text("Press \x80 to move right!", 0, 0);

Pull requests would be very welcome here!

aduros avatar Sep 06 '21 02:09 aduros

This might not be the best idea, but what about using a font file and generating the 8x8 glyphs on demand? It could generate the common glyphs at launch, some common range like the first 256 chars or something? Then just store newly generated glyphs in a texture with a coordinate lookup table. It should be simple to add commonly used glyphs to the pregen list like the arrows or popular text ui symbols.

Matt-Is-Confused avatar Sep 29 '21 18:09 Matt-Is-Confused

From what I understand one of the goals of WASM4 is to be able to one day run on microcontrollers, and such an API would be non-trivial to implement ins such a space. I think the best approach for this is to build tooling that can take

  1. A file with translation strings,
  2. A bitmap font,
  3. Source with specially marked strings,

And then build the project with the translated strings and a font image with the proper glyphs. Does that seem like a good solution?

desttinghim avatar Dec 11 '21 04:12 desttinghim

Yeah, handling text drawing yourself would be the way to go if you need characters outside of the built-in set.

I created a new issue about adding new drawing characters to the built-in font: https://github.com/aduros/wasm4/issues/195

aduros avatar Dec 13 '21 23:12 aduros