[Blocking] Flutter IME key event interception for editing implementation

[znjameswu]

Currently there is no good way of intercepting IME's key event in Flutter, e.g. Backspace, Enter, Home, End, Up, Down, etc, which is essential for the implementation of UnicodeMath-style editing.

https://github.com/flutter/flutter/issues/14809

Of course, we can infer IME key events by monitoring TextEditingValue and diffing them. But considering we are repurposing IME key events, this will either break copy/paste or undo/redo invoked from keyboard (either you get a garbled clipboard or a garbled history). It is unacceptable to break hardware keyboard shortcut just to fix IME interactions. But it is equally unacceptable to drop IME support on mobile.

We'll wait for Flutter team's solution. If anyone has experience in handling mobile IMEs on Flutter, please leave comments here.

[znjameswu]

Text diffing is not a viable option due to the following reasons according to testing.

1. Flutter Web's IME composition string implementation is broken. Composing range is always empty and any changes to the composition string are implemented by two steps: delete all & reinsert. This completely breaks text diffing method.
2. Flutter Window's has a different event model. When input N characters at once, instead of sending one TextEditingValue update, it will send N updates representing different input progress, all at the same time.

According to https://github.com/flutter/flutter/issues/50401, these discrepencies are not considered as bug.

[znjameswu]

The problem with Flutter Windows is solvable with a change of diffing algorithm.

But the problem with Flutter Web seems to be not solvable, mixing typed text with composing text means any arbitrary region of text can be reverted at any time, which makes it impossible to perform diffing to capture key event. https://github.com/flutter/flutter/issues/65357

[creativecreatorormaybenot]

@znjameswu I did not look into the issue deeply - out of curiousity: what are the problems you are facing with using RawKeyboardEventListener? Is that not a feasible approach when using TextEditingValue?

(I think you notice that I really did not look into it 😅)

[znjameswu]

@znjameswu I did not look into the issue deeply - out of curiousity: what are the problems you are facing with using RawKeyboardEventListener? Is that not a feasible approach when using TextEditingValue?

(I think you notice that I really did not look into it 😅)

When I opened this issue, I remember that RawKeyboardEventListener was unable to capture some software keyboard event on Android (i.e. complex keys from Hacker's keyboard). I just tested. It works now.

Another problem is IME handling. RawKeyboardEventListener only deals with a plain software keyboard without IME. Popping up a plain keyboard instead of user's default IME is wierd, and prevents users from inputting non-ascii characters. And I still wish to integrate this to some markdown editor in the future. So, neither constantly switching keyboard types nor dropping support for non-ascii characters seems a good idea.

It seems that RawKeyboardEventListener can partially fulfill the requirement now. But IMO this implementation would largely defeat the purpose of editing.