Support relative mouse movement

[iacore]

How to file a helpful issue

Qubes OS release

r4.1

Brief summary

RawMotion in X represents relative motion of mouse. However, the current linux client gui agent put absolute positions for the fields. It makes some games think you are moving the mouse ridiculously fast.

Steps to reproduce

1. Run xinput test-xi2 in a client VM
2. Maximize the terminal window
3. Move the mouse slowly

Expected behavior

You should see something like the following in the terminal.

EVENT type 17 (RawMotion)
    device: 2 (6)
    detail: 0
    flags: 
    valuators:
          0: 0.43 (0.43)
          1: -0.10 (-0.10)

"valuators" are (x, y) pixels moved relative to the last RawMotion event

Actual behavior

EVENT type 17 (RawMotion)
    device: 2 (6)
    detail: 0
    flags: 
    valuators:
          0: 1000.00 (1000.00)
          1: 510.00 (510.00)

"valuators" are the same as Motion "valuators". This is clearly not right.

[iacore]

RawMotion and Motion events in VM (xinput test-xi2) do not come from the same device. This means Xorg thinks there are two mice. This is also true for Button* and RawButton* events and key events. Sometimes, RawButton* and RawMotion would have the same device id.

In dom0, the mouse events have one device id, keyboard events have one device id.

[iacore]

This is definitely a driver issue, either xfree86's support for absolute pointing device is buggy, or the qubes fake device driver is somehow buggy (I can't find a issue yet).

I tried to use relative mouse motion in https://github.com/locriacyber/qubes-gui-agent-linux/tree/relative-motion, and it kind of worked. The absolute position is way off. We need to send RawMotion from dom0.

[DemiMarie]

This is definitely a driver issue, either xfree86's support for absolute pointing device is buggy, or the qubes fake device driver is somehow buggy (I can't find a issue yet).

If you can figure it out that would be amazing!

[iacore]

So, this is not a driver issue, but the application issue.

X Input documentation is really hard to find.

In https://www.x.org/releases/X11R7.7/doc/inputproto/XI2proto.txt:

A RawEvent provides the information provided by the driver to the client.

That means RawMotion may be absolute values if the device is in absolute mode.

Still, it's weird that RawMotion and Motion come from different devices in VM.

[iacore]

It would be nice to fix this, as currently VM see a pointing device like a touchscreen or touchpad, not mouse.

[DemiMarie]

It would be nice to fix this, as currently VM see a pointing device like a touchscreen or touchpad, not mouse.

Yikes! I can see how that could cause problems for some applications. On the other hand, well-behaved applications need to work with touchpads, so this is arguably a bug in those applications. Still, it’s probably worth fixing.

[iacore]

I don't have time to fix this. Directions:

• Add new events to Qubes gui protocol
• xf86 driver can switch a device between absolute & relative mode. Send one absolute event on VM window focus, then switch to relative mode. It's hard to always get absolute mouse position synced, since X was not designed for this.
• Maybe view code for some open source virtualbox-like software? They handle mouse better.

[jamke]

Wine, PCem, and maybe dosbox do work properly with mouse in Qubes OS. It is very unfortunate, because Qubes OS also has no alternative ways to run this software (unlike GNU/Linux with VirtualBox and other stuff).

Should I create separate issues or there is certainty that the reason is this issue?

[zaz]

I think @iacore's last comment is right. Here are the locations that likely need changed:

qubes-gui-daemon/gui-daemon/xside.c process_xevent_motion qubes-gui-agent-linux/gui-agent/vmside.c handle_motion qubes-gui-agent-linux/xf86-input-mfndev/src/qubes.c process_request case 'M'

And for reference, the X11 code that deals with the is_absolute argument is in xserver/hw/xfree86/common/xf86Xinput.c xf86CheckMotionEvent4DGA

[DemiMarie]

@zaz: do you know how to make these fixes?

[zaz]

@DemiMarie I don't have enough familiarity that it would be easy for me. I'd be willing to help.

The gist is to allow relative input to the X driver, then either:

1. send relative input over from dom0
2. reconstruct it in-VM from absolute coordinates

1 requires more extensive changes and sounds like the more proper way to do it. But 2 should be straightforward: just edit vmside.c to calculate deltas and send the deltas instead of the absolute mouse position. This requires no additional dom0 code or protocol changes and I'm struggling to think of any downsides over 1.

AFAIK, X does not support sending both absolute and relative positions (e.g. the cursor is at (278, 182) and just moved by (1,4).) This is a bit problematic because the conceptual model that Qubes uses is more like a touchpad from the VMs perspective. dom0 knows where the cursor is and needs to tell the VM "it is here". So theoretically, there is no nice solution to this problem. And it is possible that if we start spoofing relative movement it could cause some issue with something, but I can't think of what.

[marmarek]

Lets step back a bit and verify the actual claim in the issue:

"valuators" are (x, y) pixels moved relative to the last RawMotion event

This appears to be false. A simple test with an absolute pointing device (tried graphical tablet, on a native linux, not qubes, I guess touchscreens behave the same) shows absolute values there. And while I haven't found proper documentation for this event, the comment in the code producing those events confirms it:

https://gitlab.freedesktop.org/xorg/xserver/-/blob/master/dix/getevents.c?ref_type=heads#L1398

    /* valuators are in driver-native format (rel or abs) */

The code has some flags, especially POINTER_ABSOLUTE, but I never got that from xinput test-xi2. And by looking at the source code of it, looks like it's because it shows only "emulated" flag, but I'm not sure if those are the same flags (quick test suggests they aren't).

The real issue seems to be elsewhere - some missing flag that those are absolute values. When doing xinput list --long 15 (where 15 is that graphical tablet), I get:

...
		Detail for Valuator 0:
		  Label: Abs X
		  Range: 0.000000 - 65535.000000
		  Resolution: 0 units/m
		  Mode: absolute
		  Current value: 3966.000000
		Class originated from: 15. Type: XIValuatorClass
		Detail for Valuator 1:
		  Label: Abs Y
		  Range: 0.000000 - 65535.000000
		  Resolution: 0 units/m
		  Mode: absolute
		  Current value: 63314.000000
...

Note Mode: absolute. But on qubes VM I get:

		Class originated from: 6. Type: XIValuatorClass
		Detail for Valuator 0:
		  Label: Rel X
		  Range: -1.000000 - -1.000000
		  Resolution: 0 units/m
		  Mode: relative
		Class originated from: 6. Type: XIValuatorClass
		Detail for Valuator 1:
		  Label: Rel Y
		  Range: -1.000000 - -1.000000
		  Resolution: 0 units/m
		  Mode: relative

[marmarek]

I did grep for Relative and came up with this: https://github.com/QubesOS/qubes-gui-agent-linux/pull/228, lets see if that helps.

[marmarek]

Hmm, looking at the above xinput output, some other values might need adjustment too, for example the range...

[qubesos-bot]

Automated announcement from builder-github

The package gui-agent-linux has been pushed to the r4.3 testing repository for the Debian template. To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing bookworm-testing (or appropriate equivalent for your template version), then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package gui-agent-linux has been pushed to the r4.3 testing repository for the Debian template. To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing trixie-testing (or appropriate equivalent for your template version), then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The component gui-agent-linux (including package gui-agent-linux) has been pushed to the r4.3 testing repository for the Fedora template. To test this update, please install it with the following command:

sudo dnf update --enablerepo=qubes-vm-r4.3-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The component gui-agent-linux (including package gui-agent-linux) has been pushed to the r4.3 testing repository for the Fedora template. To test this update, please install it with the following command:

sudo dnf update --enablerepo=qubes-vm-r4.3-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The component gui-agent-linux (including package gui-agent-linux) has been pushed to the r4.3 testing repository for the Fedora template. To test this update, please install it with the following command:

sudo dnf update --enablerepo=qubes-vm-r4.3-current-testing

Changes included in this update

[marmarek]

The above change made it properly label as absolute events:

$ xinput test-xi2 6
qubesdev                                	id=6	[slave  pointer  (2)]
	Reporting 4 classes:
		Class originated from: 6. Type: XIButtonClass
		Buttons supported: 7
		Button labels: "Button Left" "Button Middle" "Button Right" "Button Wheel Up" "Button Wheel Down" "Button Horiz Wheel Left" "Button Horiz Wheel Right"
		Button state:
		Class originated from: 6. Type: XIKeyClass
		Keycodes supported: 248
		Class originated from: 6. Type: XIValuatorClass
		Detail for Valuator 0:
		  Label: Abs X
		  Range: -1.000000 - -1.000000
		  Resolution: 0 units/m
		  Mode: absolute
		  Current value: 578.000000
		Class originated from: 6. Type: XIValuatorClass
		Detail for Valuator 1:
		  Label: Abs Y
		  Range: -1.000000 - -1.000000
		  Resolution: 0 units/m
		  Mode: absolute
		  Current value: 968.000000

...

EVENT type 17 (RawMotion)
    device: 2 (6)
    time:   56565872
    detail: 0
    flags: 
    valuators:
          0: 908.00 (908.00)
          1: 1124.00 (1124.00)

But now some applications behave weird... for example selecting a word via double click in xfce4-terminal doesn't work anymore (but it does work while holding shift). And generally selecting text with mouse in xfce4-terminal is weird (if you move mouse vertically just after clicking, it scrolls instead of selecting). I think it considers this device a touchscreen, not a mouse now, and some apps change the behavior. But the Xorg log says it's still a mouse:

[     6.514] (II) Using input driver 'qubes' for 'qubesdev'
[     6.514] (**) qubesdev: always reports core events
[     6.514] (II) qubesdev: Using device /var/run/xf86-qubes-socket.
[     6.514] (**) qubesdev: always reports core events
[     6.514] (II) XINPUT: Adding extended input device "qubesdev" (type: MOUSE, id 6)
[     6.514] (II) qubesdev: num_buttons=7
[     6.514] (**) qubesdev: (accel) keeping acceleration scheme 1
[     6.514] (**) qubesdev: (accel) acceleration profile 0
[     6.514] (**) qubesdev: (accel) acceleration factor: 2.000
[     6.514] (**) qubesdev: (accel) acceleration threshold: 4
[     6.514] (II) qubesdev: On.

I'm confused...

[marmarek]

Ok, it looks like GTK is considering an input device with abs axes a touchscreen if it has ... no "mouse" in the name. So, lets rename the device to have "mouse" (or better "pointer"?)

[qubesos-bot]

Automated announcement from builder-github

The package gui-agent-linux has been pushed to the r4.3 testing repository for the Debian template. To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing bookworm-testing (or appropriate equivalent for your template version), then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package gui-agent-linux has been pushed to the r4.3 testing repository for the Debian template. To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing trixie-testing (or appropriate equivalent for your template version), then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The component gui-agent-linux (including package gui-agent-linux) has been pushed to the r4.3 testing repository for the Fedora template. To test this update, please install it with the following command:

sudo dnf update --enablerepo=qubes-vm-r4.3-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The component gui-agent-linux (including package gui-agent-linux) has been pushed to the r4.3 testing repository for the Fedora template. To test this update, please install it with the following command:

sudo dnf update --enablerepo=qubes-vm-r4.3-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The component gui-agent-linux (including package gui-agent-linux) has been pushed to the r4.3 testing repository for the Fedora template. To test this update, please install it with the following command:

sudo dnf update --enablerepo=qubes-vm-r4.3-current-testing

Changes included in this update