Put gesture injection in a separate Accessibility Service

[zacharee]

There's a weird bug in at least Samsung DeX (and possibly other Desktop Modes): having an Accessibility Service enabled that can inject gestures breaks mouse interaction with WebViews.

I'm not sure what's wrong exactly, but by separating the main Accessibility Service into two, users can choose to enable the "Tapper" service only if they need/want it.

This implementation isn't fully complete: there's no status on the home Fragment, and there isn't anything in the intro slider.

I also made it so there's room to extend the gesture functionality from only taps if desired in future versions.

This also includes commits from #585, so if it is merged, it should happen after that feature is complete.

[sds100]

are you on Android 11?

[zacharee]

Yeah. One UI 3.1 on the Galaxy Tab S7.

[sds100]

oh no, don't say its this (much bigger) Android 11 bug again.... 😒. are you sure its just accessibility services which can perform gestures that cause the problem?

[zacharee]

I removed the "canPerformGestures" flag from the main service's config XML and it let me use my mouse again. As soon as I enable a service with that flag, it kills proper mouse input.

[sds100]

can you give me an example of an app which has this issue. I'm going to test on my Android 10 S5e in dex.

[sds100]

i'd like to find another solution to this because it makes stuff more clunky and complicated since there are multiple accessibility services.

[zacharee]

Steps to reproduce:

1. Enable the main KeyMapper service (not from this commit obviously).
2. Open any browser or any app that uses a WebView renderer (reproduced in Chrome, Samsung Internet, a custom app I made that uses WebView).
3. Connect a Bluetooth or USB mouse or trackpad.
4. Attempt to interact with a webpage.

At least for me, the cursor goes wonky, showing a single cursor over the entire area of the WebView, and occasionally changing (but still applying to the whole area). If I spam-click enough, sometimes a hover event will be fired, but I can't get clicks to work.

[zacharee]

i'd like to find another solution to this because it makes stuff more clunky and complicated since there are multiple accessibility services.

You could also look at it as giving the user more granular control over what "permissions" they grant to the app. With one service, everything is granted whether it's used or not.

It does make it a little awkward, since the GestureDescription class and contained classes aren't Intent-friendly, but I implemented a mini-framework that can be extended if extra gestures are needed.

[sds100]

Yeah, it might be worth doing that. I'd like to make a more generalised solution in that case. Although, I'm wary of drastically changing stuff because a significant amount of people who use my app only know how to use it because they've watched youtube videos. if it is different to the videos then they get really confused and show the anger in bad reviews and uninstalls. 2.0 caused a dip/stagnation in new users for a month or so and a dip in the rating.

[zacharee]

That'll happen no matter what you do.

[sds100]

Yeah, I guess. Can you send a screen recording of the issue? I'm just curious to see what is happening. It doesn't happen on Android 10. I think I might split the accessibility service into 3 - key events, fingerprint gestures and performing gestures/taps.

[zacharee]

Sorry, got (very) distracted. I'll take a recording in a bit.

[zacharee]

https://drive.google.com/file/d/171QWF3YdTgMeZNjWDSAj-yIhezZheiXx/view?usp=drivesdk

[sds100]

I've sent an email requesting access to the video

[sds100]

TODO

• [ ] show an alerting notification that the user needs to turn on the new accessibility services
• [ ] move the accessibility status under the key event and fingerprint tab
• [ ] show an action error if the gesture service is disabled
• [ ] respect 'hide home screen alerts' setting

[sds100]

@zacharee are you still having this issue? I can't reproduce it on Android 11 Samsung Tab S5e in Chrome in Dex mode

[zacharee]

I'll have to try again when I get back to my tablet. I've been using a modified version with separate services this whole time. On Jul 6, 2021, 2:07 PM -0400, Seth @.***>, wrote:

@zacharee are you still having this issue? I can't reproduce it on Android 11 Samsung Tab S5e in Chrome in Dex mode —You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

[zacharee]

It looks like it might be working now. I'll have to do a bit more testing just to be sure.

[sds100]

Closing this to clean up the repo since I'll never have time to work on this. Someone can repoen it later.