Mouse input relative to the screen

[arcaine2]

Hi,

Good work on the patch, loving it, but is there any way to switch mouse input to be relative to the screen rather than to its last position? I'm trying to automate some stuff and this make it harder to predict where the current cursor can be.

Best regards.

[pelya]

Then you need to advertize as a USB graphics tablet, not as a mouse. I think Windows will support it out of the box, but you can be certain it won't work inside BIOS. Also I don't know the protocol of graphics tablets, and don't have one so cannot run it through USB analyzer, and I also don't have USB analyzer anymore.

So your best option would be to move mouse to the top-left corner using big offset, then move it slowly to the destination spot (so you don't trigger mouse acceleration, which is default in Windows). On Jul 24, 2015 10:15 PM, "arcaine2" [email protected] wrote:

Hi,

Good work on the patch, loving it, but is there any way to switch mouse input to be relative to the screen rather than to its last position? I'm trying to automate some stuff and this make it harder to predict where the current cursor can be.

Best regards.

— Reply to this email directly or view it on GitHub https://github.com/pelya/android-keyboard-gadget/issues/34.

[arcaine2]

Hmm, i see. It has to be a mouse, but it's made to control another Android device so that's something since cursor always lands in the middle of the screen everytime you connect a mouse or mouse-like HID device.

I'm struggling with one more thing tho. I'm able to move and click the mouse from within my .apk but i can't paint with it. For some reason it doesn't keep the LMB pressed. I've checked the hid-gadget-test code and it seems that the hold feature is made by sending all zeroes to USB if the mouse button is released. I recreated the same behaviour in my app but this doesn't seem to work.

public void sendMouseEvent(String key, int x, int y, boolean hold) throws InterruptedException {

    byte buffer[] = new byte[8];
    Arrays.fill(buffer, (byte) 0x00);
    try {
        FileOutputStream fos = new FileOutputStream("/dev/hidg1");
        buffer[0] = mouseFill(key);
        buffer[1] = (byte) x;
        buffer[2] = (byte) y;
        fos.write(buffer);
        if (!hold) {
            Arrays.fill(buffer, (byte) 0x00);
            fos.write(buffer);
        }
        fos.close();
    } catch (FileNotFoundException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

it's called, just for a test by something like that:

    sendMouseEvent("b1", 0, 0, true);
    sendMouseEvent("b1", 0, 50, true);
    sendMouseEvent("b1", 0, 50, true);
    sendMouseEvent("b1", 0, 0, false);

Update:

Managed to solve it by changing the bytes where the x and y is sent. What seems to work for me is:

        buffer[0] = mouseFill(key);
        buffer[5] = (byte) x;
        buffer[6] = (byte) y;

Not other changes were made. Not sure why i needed to move x and y to different byte when the first one is responsible for keeping the button pushed, but it works ;-)

[pelya]

If you want to keep button pressed, add --b1 after each coordinate:

echo --b1 | hid-gadget-test /dev/hidg1 mouse # Press the button echo 10 10 --b1 | hid-gadget-test /dev/hidg1 mouse # Move with the button pressed echo 5 -3 --b1 | hid-gadget-test /dev/hidg1 mouse # Move with the button pressed echo 5 -3 --b1 | hid-gadget-test /dev/hidg1 mouse # Move with the button pressed echo 0 0 | hid-gadget-test /dev/hidg1 mouse # Release the button

Mouse event is 4 bytes - state of buttons, X movement, Y movement, and mouse wheel movement. So you need to set buffer[0] to 1 for each event when the button is pressed. If you set byte[0] to 0 that means you released the button.

On Tue, Jul 28, 2015 at 10:32 PM, arcaine2 [email protected] wrote:

Hmm, i see. It has to be a mouse, but it's made to control another Android device so that's something since cursor always lands in the middle of the screen everytime you connect a mouse or mouse-like HID device.

I'm struggling with one more thing tho. I'm able to move and click the mouse from within my .apk but i can't paint with it. For some reason it doesn't keep the LMB pressed. I've checked the hid-gadget-test code and it seems that the hold feature is made by sending all zeroes to USB if the mouse button is released. I recreated the same behaviour in my app but this doesn't seem to work.

public void sendMouseEvent(String key, int x, int y, boolean hold) throws InterruptedException {

byte buffer[] = new byte[8];
Arrays.fill(buffer, (byte) 0x00);
try {
    FileOutputStream fos = new FileOutputStream("/dev/hidg1");
    buffer[0] = mouseFill(key);
    buffer[1] = (byte) x;
    buffer[2] = (byte) y;
    fos.write(buffer);
    if (!hold) {
        Arrays.fill(buffer, (byte) 0x00);
        fos.write(buffer);
    }
    fos.close();
} catch (FileNotFoundException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
} catch (IOException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
}

it's called, just for a test by something like that:

sendMouseEvent("b1", 0, 0, true);
sendMouseEvent("b1", 0, 50, true);
sendMouseEvent("b1", 0, 50, true);
sendMouseEvent("b1", 0, 0, false);

— Reply to this email directly or view it on GitHub https://github.com/pelya/android-keyboard-gadget/issues/34#issuecomment-125729913 .

[arcaine2]

Ok, found the real mistake. I basically copied the same code i used for sending keyboard events. It worked when coordinates were on 5th and 6th byte for some reason, but key wasn't pressed in between moves. I changed mouse buffer to 4 bytes as is should be and currently it works as expected. Fixed code:

public void sendMouseEvent(String key, int x, int y, boolean hold) throws InterruptedException {

    System.out.println("sendKeystore(): " + key);

    byte buffer[] = new byte[4];
    Arrays.fill(buffer, (byte) 0x00);
    try {
        FileOutputStream fos = new FileOutputStream("/dev/hidg1");

        buffer[0] = mouseFill(key);
        buffer[1] = (byte) x;
        buffer[2] = (byte) y;
        fos.write(buffer);
        if (!hold) {
            Arrays.fill(buffer, (byte) 0x00);
            fos.write(buffer);
        }
        fos.close();
    } catch (FileNotFoundException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
}

Thanks for your help.

[aapo]

About USB graphics tablet: I would be interested in this feature, but I need some mentoring. I have Aiptek graphics tablet for analyzing (my system seen it as Waltop).

Am I correct that patching gadget needs two things: a) Descriptor (modifications for ghid_device_android_mouse) Could this be the starting point (lsusb -d 172f:0500 -v)?

Bus 003 Device 006: ID 172f:0500 Waltop International Corp. Media Tablet 14.1"
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               1.10
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0         8
  idVendor           0x172f Waltop International Corp.
  idProduct          0x0500 Media Tablet 14.1"
  bcdDevice            1.14
  iManufacturer           1 WALTOP International Corp.
  iProduct                2 Media Tablet
  iSerial                 0 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           34
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xa0
      (Bus Powered)
      Remote Wakeup
    MaxPower              100mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         3 Human Interface Device
      bInterfaceSubClass      0 No Subclass
      bInterfaceProtocol      0 None
      iInterface              0 
        HID Device Descriptor:
          bLength                 9
          bDescriptorType        33
          bcdHID               1.10
          bCountryCode           30 Taiwan
          bNumDescriptors         1
          bDescriptorType        34 Report
          wDescriptorLength     309
         Report Descriptors: 
           ** UNAVAILABLE **
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0008  1x 8 bytes
        bInterval               2
Device Status:     0x0000
  (Bus Powered)

And does this correlate with: https://github.com/torvalds/linux/blob/master/drivers/hid/hid-waltop.c ?

b) hid-gadget-test.c for sending correct data I wiresharked graphics tablet and found that only changing values are some 8 bytes data (wireshark call it 'Leftover Capture Data').

Touching some areas gives:

near   upper left 10:00:a7:09:6e:06:00:00
near  upper right 10:00:ca:37:14:03:00:00
near  bottom left 10:00:38:08:58:3a:00:00
near bottom right 10:00:5f:38:d6:3a:00:00

Which lead me to guess:

Offset    Value
  0       10 (so far for every packet)
  1       00=release / 10=moving near the surface / 11=pressed  / 12=secondary button pressed
  2       ??  (Maybe tilting?)
  3       00 ... 3f (x-coord, growing from left to right)
  4       ??  (Maybe tilting?)
  5       00 ... 3f (y-coord, growing from top to bottom)
  6,7     pressure. 0000 if not pressed. (Maximum I saw 99 03, where last part is more significant)