Emoji input via virtual keyboard produces error

[thomas-ah]

Description

When inputting an emoji into Mirage through the virtual keyboard the following error pops up in the JS debug pane:

Unexpected error occured: UnicodeEncodeError. Traceback (most recent last):
  File "qrc:/src/backend/user_files.py", line 105, in _write_loop
    await new.write(self._to_write)
  File "/usr/lib/python3.9/site-packages/aiofiles/threadpool/utils.py", line 36, in method
    return (yield from self._loop.run_in_executor(self._executor, cb))
  File "/usr/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
UnicodeEncodeError: 'ascii' codec can't encode character '\U0001f600' in position 617: ordinal not in range(128)

Your environment

• OS or distribution: Manjaro Plasma-mobile (Arch linux)
• Architecture: ARM64
• Desktop environment: Plasma-mobile (Wayland)
• Installed method: automatically through AUR, matrix-mirage package (installed version 0.6.4-2)

Steps to reproduce

1. Open Mirage, open chat room
2. Select bottom text input field to make virtual keyboard popup (if it did not do so automatically already)
3. Switch virtual keyboard to emoji-input
4. Press an emoji to input, any emoji will trigger the error

Expected behavior

The emoji should be inserted into the text input field without error.

Actual behavior

The emoji is inserted into the text field and the above-mentioned error also appears.

Extra note

As described the emoji does appear in the text input field, and therefore the functionality does work as expected. Since the error can be hidden by checking the 'Hide for this session' checkbox the bug is not very grave in nature, and is not actually blocking in any manner.

[mirukana]

When you type in the composer, it saves the text in a file at $XDG_DATA_HOME/mirage/state.json (if that variable is set) or ~/.local/share/mirage/state.json, so that it can be restored if you switch room and come back or the app crashes in the middle of typing. It seems like this file is not being opened in UTF-8 mode on your system for some reason.

Can you paste the code below into a file on the device, run it with python3 <file> and report the result?

import os

for var, value in os.environ.items():
    if var == "LANG" or var.startswith("LC_"):
        print(var, "=", value)

with open(os.path.expanduser("~/.local/share/mirage/state.json")) as file:
    print(file.encoding)

with open("/tmp/test123", "w") as file:
    print(file.encoding)

with open("/tmp/test123", "w", encoding="utf-8") as file:
    print(file.encoding)

print("\U0001f600")

[yogo1212]

@mirukana i'm also running plasma on manjaro.

this is the output for me:

[kde@pinephoen ~]$ python /tmp/muh.py
LANG = C
UTF-8
UTF-8
utf-8
😀

so, that should be fine. sadly, i'm not able to test the error messages since i'm still unable to start mirage :-( it justs loops with a circle. state.json exists already, though.

[EDIT: got it to work]

[thomas-ah]

Hey @mirukana, thanks responding and apologies for my late reply :) I've tested your piece of code, the input was the same for me as for @yogo1212:

$ python python_test.py
LANG = en_US.UTF-8
UTF-8
UTF-8
utf-8
😀

However, something changed (in Plasma Mobile itself probably since its being developed in a very rapid pace), because currently I'm not able to input emoji's into Mirage through the virtual keyboard at all. The error I mentioned in this ticket is also no longer triggered and therefore I'm unable to reproduce the problem... I think this issue can be closed, and perhaps reopened when I can reproduce it.

Again, thanks for wanting to look into it though :)

[mirukana]

This happens outside of plasma mobile too from #222, still not sure what's going on. Another script to try, paste in a file and run with python3 <filepath>:

import asyncio
import aiofiles

async def main():
    async with aiofiles.open("/tmp/abc", "w") as file:
        await file.write("\U0001f600")

    async with aiofiles.open("/tmp/abc", "r") as file:
        print(await file.read())

asyncio.get_event_loop().run_until_complete(main())