Scrcpy interaction with OPPO Phone is not working

[EloZ27]

I have tried scrcpy and various guidelines for mirroring phones via scrcpy but it seems that the interaction with keyboard and mouse and audio does not work with my OPPO F11.

Help

[rom1v]

https://github.com/Genymobile/scrcpy/blob/master/FAQ.md#mouse-and-keyboard-do-not-work

?

[EloZ27]

Yes, I have already read that. But based in my phone I don't have such option for that. I even looked entirely in my phone, but I haven't found any option for it.

[rom1v]

Any hint in adb logcat?

What happens if you focus some text area on your device, and execute adb shell input text hello from your computer?

[EloZ27]

Cant seem to find any hint in the logcat.

adb shell input text hello worked

[rom1v]

adb shell input text hello worked

Oh, that's very weird, then. Pressing h e l l o in the scrcpy windows does not work?

What if you start with scrcpy --prefer-text?

[EloZ27]

Hello. Here is the result for the scrcpy --prefer-text.

[rom1v]

But what happens when you enter text in the scrcpy window?

[EloZ27]

Hmmm. Somehow, typing in the keyboard worked in scrcpy. But the mouse still does not interact

[rom1v]

Probably similar to #1347.

Does this work: https://github.com/Genymobile/scrcpy/issues/1347#issuecomment-624039383

[EloZ27]

Yea. It some how tapped my screen.

[rom1v]

Please test with older versions of scrcpy, if you find one which work, it will help a lot to find the cause.

[EloZ27]

I have tried versions down to v1.4, scroll wheel and left mouse button does not work. Middle click and right click worked as well as the keyboard.

[GetCurious]

scrcpy 1.12.1

dependencies:

• SDL 2.0.10
• libavcodec 58.54.100
• libavformat 58.29.100
• libavutil 56.31.100

adb shell input tap 179 179

log.txt

same phone (F11 Pro), same situation, most keyboard, mid click & right click works except left click.

[rom1v]

adb shell input tap 179 179

Does it work or not?

[GetCurious]

now running build master version

scrcpy 1.14

dependencies:

• SDL 2.0.10
• libavcodec 58.54.100
• libavformat 58.29.100
• libavutil 56.31.100

the command works. but left click doesnt.

[rom1v]

Oh, could you test with this change, please:

diff --git a/app/src/input_manager.c b/app/src/input_manager.c
index 9c22ee0a..7d500efe 100644
--- a/app/src/input_manager.c
+++ b/app/src/input_manager.c
@@ -511,7 +511,8 @@ convert_mouse_button(const SDL_MouseButtonEvent *from, struct screen *screen,
     to->inject_touch_event.position.screen_size = screen->frame_size;
     to->inject_touch_event.position.point =
         screen_convert_window_to_frame_coords(screen, from->x, from->y);
-    to->inject_touch_event.pressure = 1.f;
+    to->inject_touch_event.pressure =
+        from->type == SDL_MOUSEBUTTONDOWN ? 1.f : 0.f;
     to->inject_touch_event.buttons =
         convert_mouse_buttons(SDL_BUTTON(from->button));
 

[GetCurious]

convert_mouse_button(const SDL_MouseButtonEvent *from, struct screen *screen,
                     struct control_msg *to) {
    to->type = CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT;

    if (!convert_mouse_action(from->type, &to->inject_touch_event.action)) {
        return false;
    }

    to->inject_touch_event.pointer_id = POINTER_ID_MOUSE;
    to->inject_touch_event.position.screen_size = screen->frame_size;
    to->inject_touch_event.position.point =
        screen_convert_to_frame_coords(screen, from->x, from->y);
    to->inject_touch_event.pressure =
        from->type == SDL_MOUSEBUTTONDOWN ? 1.f : 0.f;
    to->inject_touch_event.buttons =
        convert_mouse_buttons(SDL_BUTTON(from->button));

    return true;
}

no difference. can i have your full input_manager.c?

[rom1v]

no difference.

Could you please test this change instead (or in addition), on dev:

diff --git a/server/src/main/java/com/genymobile/scrcpy/Controller.java b/server/src/main/java/com/genymobile/scrcpy/Controller.java
index 71e7ec9c..1f249225 100644
--- a/server/src/main/java/com/genymobile/scrcpy/Controller.java
+++ b/server/src/main/java/com/genymobile/scrcpy/Controller.java
@@ -11,7 +11,7 @@ import java.io.IOException;
 
 public class Controller {
 
-    private static final int DEVICE_ID_VIRTUAL = -1;
+    private static final int DEVICE_ID_VIRTUAL = 0;
 
     private final Device device;
     private final DesktopConnection connection;

can i have your full input_manager.c?

(I use the one from current dev branch)

Note: adb shell input tap ... calls this: https://github.com/aosp-mirror/platform_frameworks_base/blob/22e3e74e4b3b989d20bb17ca7d54f95b67d6c02c/cmds/input/src/com/android/commands/input/Input.java#L221-L226

[GetCurious]

i got nothing (instead and in-addition).

[rom1v]

@GetCurious OK, just to be sure, you are building the server (you don't use the prebuilt-server)?

What we have to understand is why this works (adb shell input tap ...): https://github.com/aosp-mirror/platform_frameworks_base/blob/22e3e74e4b3b989d20bb17ca7d54f95b67d6c02c/cmds/input/src/com/android/commands/input/Input.java#L221-L226

but this does not: https://github.com/Genymobile/scrcpy/blob/5fa46ad0c71169f9369f9f3bde9cce3d29ed4b88/server/src/main/java/com/genymobile/scrcpy/Controller.java#L198-L200

[GetCurious]

$ cd scrcpy
$ vim app/src/input_manager.c
$ vim server/src/main/java/com/genymobile/scrcpy/Controller.java
$ meson x --buildtype release --strip -Db_lto=true
$ ninja -Cx 
$ ./run x

anything i miss?

[rom1v]

Let's try to use a device id matching the input source, like they do:

diff --git a/server/src/main/java/com/genymobile/scrcpy/Controller.java b/server/src/main/java/com/genymobile/scrcpy/Controller.java
index 71e7ec9c..c1949f36 100644
--- a/server/src/main/java/com/genymobile/scrcpy/Controller.java
+++ b/server/src/main/java/com/genymobile/scrcpy/Controller.java
@@ -195,12 +195,26 @@ public class Controller {
             }
         }
 
+        int deviceId = getInputDeviceId(InputDevice.SOURCE_TOUCHSCREEN);
+        Ln.i("device id = " + deviceId);
         MotionEvent event = MotionEvent
-                .obtain(lastTouchDown, now, action, pointerCount, pointerProperties, pointerCoords, 0, buttons, 1f, 1f, DEVICE_ID_VIRTUAL, 0,
+                .obtain(lastTouchDown, now, action, pointerCount, pointerProperties, pointerCoords, 0, buttons, 1f, 1f, deviceId, 0,
                         InputDevice.SOURCE_TOUCHSCREEN, 0);
         return device.injectEvent(event);
     }
 
+    private static int getInputDeviceId(int inputSource) {
+        final int DEFAULT_DEVICE_ID = 0;
+        int[] devIds = InputDevice.getDeviceIds();
+        for (int devId : devIds) {
+            InputDevice inputDev = InputDevice.getDevice(devId);
+            if (inputDev.supportsSource(inputSource)) {
+                return devId;
+            }
+        }
+        return DEFAULT_DEVICE_ID;
+    }
+
     private boolean injectScroll(Position position, int hScroll, int vScroll) {
         long now = SystemClock.uptimeMillis();
         Point point = device.getPhysicalPoint(position);

[GetCurious]

no difference

[rom1v]

What's the id printed in the console when you click with this change?

[GetCurious]

how do i do that?

[rom1v]

When you execute ./run x, what is the output in the console?

[GetCurious]

➜  scrcpy git:(dev) ✗ ./run x                                                      
INFO: scrcpy 1.14 <https://github.com/Genymobile/scrcpy>
x/server/scrcpy-server: 1 file pushed, 0 skipped. 107.0 MB/s (33374 bytes in 0.000s)
[server] INFO: Device: OPPO CPH1969 (Android 10)
INFO: Renderer: opengl
INFO: OpenGL version: 4.6.0 NVIDIA 440.100
INFO: Trilinear filtering enabled
INFO: Initial texture: 1080x2336
INFO: New texture: 888x1920

[rom1v]

And nothing more, even when you click in the scrcpy window?

EDIT: in itself:

INFO: Initial texture: 1080x2336
INFO: New texture: 888x1920

it's a bit surprising (the device sends frame at a different resolution from what has been requested). But probably not related.

[GetCurious]

that's right... nothing more. everything except left-click works

[rom1v]

With that change: https://github.com/Genymobile/scrcpy/issues/1518#issuecomment-655440431 it should, since a log is added on every click.

Unless the client does not send the events to the device. Let's add more logs:

diff --git a/app/src/input_manager.c b/app/src/input_manager.c
index 7d500efe..fc0edb6d 100644
--- a/app/src/input_manager.c
+++ b/app/src/input_manager.c
@@ -523,6 +523,14 @@ void
 input_manager_process_mouse_button(struct input_manager *im,
                                    const SDL_MouseButtonEvent *event,
                                    bool control) {
+    LOGI("mouse button: type=%d, which=%d, button=%d, state=%d, clicks=%d, x=%d, y=%d",
+            (int) event->type,
+            (int) event->which,
+            (int) event->button,
+            (int) event->state,
+            (int) event->clicks,
+            (int) event->x,
+            (int) event->y);
     if (event->which == SDL_TOUCH_MOUSEID) {
         // simulated from touch events, so it's a duplicate
         return;
@@ -559,9 +567,12 @@ input_manager_process_mouse_button(struct input_manager *im,
 
     struct control_msg msg;
     if (convert_mouse_button(event, im->screen, &msg)) {
+        LOGI("==== convert_mouse_button OK");
         if (!controller_push_msg(im->controller, &msg)) {
             LOGW("Could not request 'inject mouse button event'");
         }
+    } else {
+        LOGW("==== convert_mouse_button KO");
     }
 }