tinypilot icon indicating copy to clipboard operation
tinypilot copied to clipboard

Support a "force QWERTY" mode

Open mtlynch opened this issue 3 years ago • 0 comments

Problem

TinyPilot sends keyboard input by forwarding key events by sending the KeyboardEvent.code value for the input event. The the KeyboardEvent.code is agnostic to keyboard layout, as it represents the physical position of the key on the keyboard.

Sending the KeyboardEvent.code value means that the target computer gets to decide how to interpret the key depending on the layout of the target system. The behavior is equivalent to if your keyboard was directly plugged into the target system.

In some cases, TinyPilot's default behavior is not desirable. If a TinyPilot user has a keyboard with the NEO layout, but their target system has a QWERTY layout, the keys the user presses as they observe on their local keyboard won't match the keys that appear on the target system.

For example, on a NEO local system and a QWERTY target system:

  1. User pushes the button labeled h on their NEO keyboard
  2. TinyPilot receives JS keystroke with event.code='KeyU'
  3. TinyPilot translates KeyU to HID KEYCODE_U=0x18
  4. A 'u' character appears on the target system

If the user actually meant for the target system to receive the h keystroke, then TinyPilot's behavior is undesirable.

This scenario can happen when the target system is in an OS environment without keyboard layout options, so it assumes QWERTY.

Proposed solution

TinyPilot should offer a configuration option where the user can elect to "force QWERTY" where TinyPilot assumes that the target system is QWERTY and forces keystrokes to their QWERTY equivalent.

When "force QWERTY" mode is active, instead of looking at the KeyboardEvent.code property, we'd look at KeyboardEvent.key, which would represent the layout-aware character they pressed. We'd then translate that key into the equivalent HID code, assuming the target system is QWERTY.

For example, on a NEO local system and a QWERTY target system (with "force QWERTY" enabled):

  1. User pushes the button labeled h on their NEO keyboard
  2. TinyPilot receives JS keystroke with event.key='h'
  3. TinyPilot translates h to HID KEYCODE_H=0x0b
  4. An 'h' character appears on the target system

mtlynch avatar Oct 31 '22 20:10 mtlynch