Adding a Feature: KeyToTouch

[nothingstopsme]

Hi, I have just added a new feature "KeyToTouch", which allows users to initiate a touch event via pressing some key on a keyboard.

My initial motivation was to make the direct manipulation of buttons of android GUI through key events possible, in an attempt to use an android Input Method Editor (IME) for typing texts. While in terms of controlling IME this is far from perfect and less straightforward to users, this implementation is thought to be general and therefore it can serve as "hot key" utility for any similar purposes.

I hope this can be useful for other users, and let me know if there are any problems.

Cheers.

[srevinsaju]

hello, how can you set keys and map them to touch points?

[nothingstopsme]

Hi, it is just a super simple method (probably a bit inconvenient for some users :) )

I have defined a xml file (key-to-touch-map.xml) which users can edit and put their desired mapping inside. That xml will be adb pushed along with the server file (scrcpy-server) to a connected device at startup, and the server will read it to initialise the mapping rules.

I have also put some mappings I used for testing into the xml file I committed as an example. You might want to take a look at it if you are interested in the format. Thank you.

[srevinsaju]

@nothingstopsme I had the same idea too, but it was not intuitive to users to select points on the screen and decide the keys to work. the Backend is perfect in your code. this is a similar implementation of the key to touch implementation, but with GUI. (Very basic gui, still wondering if people use them ;) ).

https://github.com/srevinsaju/guiscrcpy/blob/master/guiscrcpy/mapper.py

try running it with

python3 mapper.py

and maybe you can add more commits to include the GUI part too to scrcpy. anyway good work!

[barry-ran]

You can see the implementation of QtScrcpy, without changing the server, it can already use the mouse and keyboard to play games

support custom keymap，like this