[Bug]Unable to use UnicodeMap with WinCompose under Windows.

[Piervit]

Describe the Bug

I am playing on a qmk customized firmware (for keychron Q1 V1 ISO keyboard). I mostly want to rebind key to send unicode values (such as "ŭ" (esperanto keyboard)).

I have followed https://docs.qmk.fm/features/unicode, I use in my rules.mk file:

UNICODEMAP_ENABLE = yes

And in keymap.c, I use UP keycode like this: UP(EO_UA,EO_UAU) with EO_UA and EO_UAU:

[EO_UA] = 0x016D, // ŭ [EO_UAU] = 0x016C, // Ŭ

Under Linux (using UNICODE_MODE_LINUX), I am fully happy about it.

Under Windows (using the advised UNICODE_MODE_WINCOMPOSE), that is not working as expected. On my windows computer, I have installed the wincompose software and set it accordingly. Here is what I get when I send my unicode key:

What I see is that my compose action is well understood however It fails on "Invalid sequence: "uà"" message. My understanding is that WinCompose do not expect u + raw UTF8 code (as sent by qmk keyboard) but would like u + 1 + 6 + D. This is looks conform with the Wincompose doc (https://github.com/samhocevar/wincompose) but not with what is done by qmk.

I am not able to be sure if the bug is on QMK side or on WinCompose. Or I am missing something?

Keyboard Used

No response

Link to product page (if applicable)

No response

Operating System

No response

qmk doctor Output

Is AutoHotKey / Karabiner installed

• [ ] AutoHotKey (Windows)
• [ ] Karabiner (macOS)

Other keyboard-related software installed

No response

Additional Context

No response

[sigprof]

What input language and keyboard layout are you using in Windows?

According to the VK codes, QMK is sending KC_U, KC_0, KC_1, KC_6, KC_D ("u016d"); however, Windows interprets KC_0 as à, KC_1 as &, KC_6 as -. Maybe the keyboard layout which you are using in Windows has such mappings. In this case your options are:

• Switch the input language and keyboard layout in Windows to something which is compatible enough with US QWERTY (with the default QMK configuration the Unicode input support requires that all of the 0123456789abcdef characters are in the same positions as in the US QWERTY layout; KC_U for UNICODE_MODE_WINCOMPOSE may be redefined by overriding unicode_input_start()).
• Configure QMK to use language-specific mappings for SEND_STRING() (the Unicode support code uses send_nibble(), which goes through those mappings). But this option cannot be switched at runtime, so you would probably have difficulties with Linux unless you also reconfigure the keyboard layout there to match the layout used by Windows.

[Piervit]

ho, you are fully right Sigprof. My layout is french azerty. I am both happy to understand more and disappointed: I though numeric keypad and keyboard number where different and that when using that unicodes chars, they were translated to numeric keypad and so independant from layout. Moreover my linux layout is also french azerty layout and I did not experience that issue. Both solution you give makes me quite unhappy(I though I could have unicode independantly of os layout nor specific language stuff), Anyway thanks you a lot sigprof for your explanation!

I wait a few days, in case someone else come with a better solution (or an explanation about why numeric keypad cannot be used to be layout independant) and I will close issue (as it might not be improvable).

[sigprof]

The problem with numeric keypad is that its behavior depends on the Num Lock state at least in Windows. The code for UNICODE_MODE_WINDOWS has some special handling for that (it looks at the Num Lock indicator state and toggles Num Lock on before sending the sequence), but the code for UNICODE_MODE_WINCOMPOSE does not do that. However, another problem is the a character, the keycode for which also would be different in AZERTY and QWERTY; so even switching to the numeric keypad for digits won't fix the problem completely.

Not sure why you did not notice any problems with Linux — there are reports like ibus/ibus#2645 which specifically mention AZERTY (although this mostly deals with the a character, which you might not have in the Unicode values you tried to use).

In general, the situation with Unicode input sucks in all OSes (there are some complaints even about the macOS implementation, which is the only one that could be configured without any third-party software or things like the registry hack which is needed to enable UNICODE_MODE_WINDOWS).

[Piervit]

Sigprof, I really appreciate your help. I should offer you a drink.

In my case, I don't the a character (even if I agree that it means a problem in the general case).

I have quickly tried the following change to mimick the keypad behaviour of UNICODE_MODE_WINDOWS with WinCompose. After a quick test it appears to work (except when I use a GTK app under windows....), I will check a bit more.

diff --git a/quantum/unicode/unicode.c b/quantum/unicode/unicode.c
index dff1d43fb4..0f10b29b0f 100644
--- a/quantum/unicode/unicode.c
+++ b/quantum/unicode/unicode.c
@@ -239,6 +239,9 @@ __attribute__((weak)) void unicode_input_start(void) {
             tap_code(KC_KP_PLUS);
             break;
         case UNICODE_MODE_WINCOMPOSE:
+            if (!unicode_saved_led_state.num_lock) {
+                tap_code(KC_NUM_LOCK);
+            }
             tap_code(UNICODE_KEY_WINC);
             tap_code(KC_U);
             break;
@@ -312,7 +315,7 @@ __attribute__((weak)) void unicode_input_cancel(void) {
 // clang-format off
 
 static void send_nibble_wrapper(uint8_t digit) {
-    if (unicode_config.input_mode == UNICODE_MODE_WINDOWS) {
+    if (unicode_config.input_mode == UNICODE_MODE_WINDOWS || unicode_config.input_mode == UNICODE_MODE_WINCOMPOSE) {
         uint8_t kc = digit < 10
                    ? KC_KP_1 + (10 + digit - 1) % 10
                    : KC_A + (digit - 10);

[Piervit]

I will close the issue with this comment as Sigprof responded correctly to it.

QMK was sending the correct code, but they were interpreted by my keyboard layout. I finally success using numeric keypad, as long as I am lucky enough I don't have to send the A hexa character.

I did not explore SEND_STRING() for now .