WSA
WSA copied to clipboard
microsoft
Using the touchpad with two fingers side by side
Is your feature request related to a problem? Please describe
Using the touchpad with two fingers side by side generates multiple directional scroll gestures. This behavior does not exploit the precision that this input device should allow. This is particularly annoying when you move around in a document.
Describe the solution you'd like
I think the behavior should be the same as on chromebooks, i.e. click and drag.
Describe alternatives you've considered
No response
Please specify the version of Windows Subsystem for Android
2208.40000.5.0
We'll investigate and follow up on this. Thank you for bringing this to our attention.
Could you provide more details about what app you're using and what the expected behavior is?
For context, WSA intentionally does not pass raw input through to Android since that can cause user experience issues (e.g., Windows and Android would have different views of the mouse cursor position) and application compatibility issues (e.g., many Android apps don't support mouse input). Instead, WSA reports the mouse cursor position computed by Windows and translates mouse scroll to touch panning. It is expected that two-finger touchpad scroll gestures are translated to touch pan gestures.
We have been refining our emulated pan gestures to improve scrolling smoothness, so if you have a specific app that isn't scrolling well, please share the details so we can investigate. We also have an item on our backlog to disable the emulated touch scrolling for apps that declare the android.hardware.type.pc feature, in which case these apps would receive MotionEvent.ACTION_SCROLL input events. However, most apps don't take advantage of this feature. If an app requires two-finger swipe gestures, you can perform a two-finger free-form gesture with the mouse by holding down ALT+left mouse button.
Thank you for this information, I understand what you are saying and I have no problem with mouse or touchpad actions being translated into a touch event. I'm more concerned with how it's done.
As it is implemented, it is as if you were using a D-PAD, you have no precision. The touch pad is underused. I can't go in the direction I want, it's either horizontal or vertical. When I move my fingers I can't stop where I want to, and when I pull my fingers out it keeps moving.
Using two fingers on a touchpad and clicking with the mouse to move a view have in common that it is the most natural way to move a view. It's about user experience. Indeed, clicking a touchpad frequently is painful. The source device that generates an event is important, the behaviors are not interchangeable in terms of user experience.
My application is CDisplayEx, it's a comics reader currently available on amazon store. Version 1.3.33 does not yet include any WSA adaptations. Version 1.3.34 intercepts artificially generated gestures using the touch event flag. Version 1.3.34 is being deployed but is slow to arrive.
Well, my problem concerns only the reader interface. If you use two fingers, it creates horizontal or vertical jumps. Indeed if I look at the events, I have several times this sequence of events (ACTION_DOWN + ACTION_MOVE + ACTION_UP) over a certain distance. These events are identified by this flag: MotionEvent.FLAG_IS_GENERATED_GESTURE. This flag allows me to intercept these gestures and have a less unpleasant behavior that follows the normal reading flow of the book. But I'm not satisfied, it should generate only one movement of the exact length of the movement made with the fingers (ACTION_DOWN + ACTION_MOVE + ... + ACTION_MOVE + ACTION_UP). This should allow me to detect the velocity of the movement and continue the movement if necessary. Like with a mouse.
If I may, this problem also seems to be related to the behavior of the mouse wheel, the wheel is imprecise, it seems normal to me, but again I have to intercept the flag to hijack this behavior in order to follow the reading flow which is not necessarily vertical but also horizontal. WSA hides the existence of the mouse from the application, again the origin of the event is important.
Thanks for the detailed explanation. Windows automatically converts touchpad two-finger gestures to WM_MOUSEWHEEL/WM_MOUSEHWHEEL input messages, so WSA doesn't distinguish between touchpad and mouse scroll and has no knowledge of diagonal gestures. It's possible for WSA to opt-out of Windows's automatic conversion and receive raw touchpad input, but then we will no longer receive WM_MOUSEMOVE for touchpad either. WSA would either have to pass through the raw touchpad input to Android, in which case the mouse cursor position would be out-of-sync with Windows, or WSA would have to perform its own gesture detection to identify cursor movement vs. scroll vs. pinch, which is duplicating what we currently get for free from Windows. I don't think that's an approach we want to take.
If it would be easier for your app to handle MotionEvent.ACTION_SCROLL containing the same data WSA received from Windows in WM_MOUSEWHEEL/WM_MOUSEHWHEEL rather than trying to interpret our emulated gestures, we can easily do that for apps that declare the android.hardware.type.pc feature (which indicates the app expects mouse/keyboard input). We already have the code to inject MotionEvent.ACTION_SCROLL, but we disabled it when adding the touch emulation. We can use that code instead of touch emulation for apps that declare android.hardware.type.pc. We can probably ship that early next year.
If the repeated ACTION_UP events are the main issue with interpreting the emulated gestures, I can experiment with leaving the emulated touch pointer down for multiple seconds after the last scroll event or until some other input cancels it. Then you should only see ACTION_UP if the emulated pointer reaches the edge of your window (when we reset the pointer position). Whether we can ship that or not depends on the results of app compatibility testing since some apps may not handle a lingering pointer well. Would that be useful?
Thank you for these explanations, I understand better what are your constraints.
If WSA generates ACTION_SCROLL events when it receives the WM_MOUSEWHEEL message, then the event will be better identified and I won't need my hack anymore. I'm ok with that and I will use the android.hardware.type.pc feature.
Avoiding repeating ACTION_UP will not solve my main problem unfortunately. If I understand correctly, it can't be solved until you use raw data or Windows offers more advanced touch pad management comparable to MacOS or ChromeOS.
Nevertheless, I think you are doing an excellent job with WSA.
Make sure to declare android.hardware.type.pc as required="false" so your app will still install on phones/tablets. :)
I'm having similar issues in my book-type app where I detect swipes with calls to onFling in Gesture Detector Listener to flip the pages.
On Windows a two finger swipe generates multiple calls to onFling in a GestureDetector.SimpleOnGestureListener including extra ones even after the fingers have been lifted from the touchpad. (Maybe to simulate inertia?) On Chromebooks we only get a single onFling call for each two finger swipe, which nicely mimics a single finger swipe on touch devices.
As suggested by @hgourvest, I made a workaround looking for MotionEvent.FLAG_IS_GENERATED_GESTURE and using a timestamp to discard events that come in rapid succession after the initial one, but this is somewhat limiting because of the extra simulated "inertia" events that keep coming in even after the finger motion has ended. The user has to wait a second or so, for the "inertia" events to stop before a new swipe can be detected.
Is there a way to differentiate the first generated MotionEvent of a two finger swipe? I could use that to reset the timestamp instead of waiting for a certain amount of time of "silence".
Each onFling event has a start event and an end event (e1 & e2). Between two onFling events (F1 & F2), I measure the time between e2(F1) and e1(F2) with the MotionEvent.getEventTime function, the duration is very short and the precision is 63ms, so I test if the duration is longer than 2x63ms to trigger the event. I get a good result, I hope it will be useful to you.
Thank you for the suggestion @hgourvest. Yes, this helps discarding the extra motion events, but only if you let enough time pass between finger swipes. You cannot make multiple swipes at a natural "page-flipping" pace (2-3 pages a second when browsing around). Each user swipe is followed by 3-9 system generated events (many after the fingers have come off the touchpad), so if the user follows the first swipe with a quick second swipe, its motion events will be mixed-in with the tail of generated events from the previous swipe. When you swipe multiple times at a medium/fast pace, all the event's delta measure the same ~63ms, so in this situation, I can't think of a way to distinguish new user-initiated swipes form the system generated ones.
yes @massimobio, this is not the ideal solution, we will have to wait for the next version of WSA and use the "android.hardware.type.pc" feature. You can already prepare your code by capturing the MotionEvent.ACTION_SCROLL event in the "onGenericMotionEvent" method and which is triggered with the mouse wheel on a normal version of android. Shift + Wheel should generate what will happen with a horizontal swipe.
I checked in the "android.hardware.type.pc" opt-out for touch-emulation today, so MotionEvent.ACTION_SCROLL will be available in WSA's 2212 build, which should ship in early 2023.
I believe the multiple onFling gestures is because WSA may inject multiple emulated pan gestures for scroll input (including touchpad scroll) if there is more scroll delta than can fit between the mouse cursor position and the window edge. When the emulated touch pointer reaches the window edge, we reset it to the mouse cursor position and inject another pan gesture if there is remaining scroll delta. Only injecting a single pan gesture would avoid the multiple onFling gestures, but it would cause most apps to scroll unacceptably slow because we would drop part of the scroll delta. This is especially noticeable with mice with fast/free-spinning scroll wheels. Hopefully, using MotionEvent.ACTION_SCROLL is sufficient for you.