engine
engine copied to clipboard
[WIP] Synthesize key events for shift key on pointer events.
This WIP PR is an attempt to implement a way to synchronize modifier keys state based on pointer data. This is a web only implementation to discuss the implementation choice. It might leads to non-WIP PRs for each platforms.
What problem does it solve?
Engine and framework communicate to maintain information about the current keyboard state. When a Flutter window lost the main focus, it has no way to be notified of modifier keys state changes which leads to possible bugs such as https://github.com/flutter/flutter/issues/112488.
Implementation
The implementation is based on @dkwingsmt insights from https://github.com/flutter/flutter/issues/112488#issuecomment-1271892594 : "
- When the engine receives a pointer event, extract its keyboard state, and send it to the keyboard manager.
- The keyboard manager synchronizes with the states, dispatch synthesized events as needed.
- If this procedure is done synchronously, the synthesized key events will arrive at the framework before the pointer events, thus correctly affecting the next pointer event.
Notes:
- This works the best for the hardware keyboard system.
- If we really want to support the (to be deprecated) raw keyboard system, the raw keyboard manager can add the key state to the raw key data JSON payload, and process synchronization on the framework side.
"
Hey @dkwingsmt ! Before going further (implementing support for all modifier keys and add tests), I'm very interested by your feedback on this work in progress. For the moment:
- only shift key is handled (I will refactor this to handled other modifier keys).
- on each pointer event we check if the known shift key state has changed.
- if the state has changed, a key up or down event is synthesized. I manually tested this on https://github.com/flutter/flutter/issues/112488 and it seems to fix it.
Good job! Everything so far looks great. Thank you for working on it.
Thanks!
Another task is to support all 3 event adapters:
PointerEvent,MouseEvent, andTouchEvent. For now you only supported the first.
While implementing this, I noticed that TouchEvent modifiers accesors are not defined. I filed a PR to add them, thank you to review it: https://github.com/flutter/engine/pull/36836.
@dkwingsmt I have updated the PR mainly to add tests.
To make testing easier and make the dependence between PointerBinding and KeyboardBinding more explicit, I choose to inject the KeyboardConverter in PointerBinding.initInstance.