Allow temporary VM mouse capture with user consent

[DemiMarie]

Qubes OS version Qubes release 4.0 (R4.0)

Affected component(s) or functionality Mouse virtualization

Brief summary Minecraft is not playable in an AppVM. The graphics work fine, but I can’t aim the camera.

Others have reported no such problems, and I believe that this may relate to QubesOS’s mouse virtualization, so I am reporting this as a QubesOS bug

To Reproduce

1. Create a new Debian-based TemplateVM or StandaloneVM
2. Purchase Minecraft (Java edition)
3. Install the official Debian package for Minecraft.
4. Try to play

Expected behavior It works, albeit with slow frame rate due to the lack of hardware-accelerated graphics.

Actual behavior I cannot get the camera to move vertically in any controllable way, rendering the game unplayable.

Screenshots I don’t have any, but can provide them if they would be helpful.

Additional context This could be an issue with the Linux touchpad drivers, but I doubt it.

Solutions you've tried USB pass-through of a USB mouse.

[zaoqi]

Minetest also has this problem.

1. Create a Fedora 30-based VM.
2. $ dnf install minetest
3. $ minetest

[biergaizi]

I suggest removing "for full-screen applications" from the title - allowing mouse capture is also useful in non-fullscreen applications. For example, CAD packages, such as KiCad or LibreCAD, heavily relies on mouse capture for navigation. Without a mouse capture, it's almost impossible to use any 2D CAD programs in Qubes.

[tslilc]

I hope this is not an inappropriate place to ask, but where are these safeguards implemented so that i might have a go at understanding the current implementation?

[DemiMarie]

I hope this is not an inappropriate place to ask, but where are these safeguards implemented so that i might have a go at understanding the current implementation?

The current code simply does not support mouse capture.

[DemiMarie]

I suggest removing "for full-screen applications" from the title - allowing mouse capture is also useful in non-fullscreen applications. For example, CAD packages, such as KiCad or LibreCAD, heavily relies on mouse capture for navigation. Without a mouse capture, it's almost impossible to use any 2D CAD programs in Qubes.

What do these applications use under Wayland?

[DemiMarie]

Update: At least Minetest is quite playable under Qubes OS. It is somewhat slow, but that appears to be because of server performance limitations, not software rendering.

[biergaizi]

@DemiMarie: What do these applications use under Wayland?

Interesting question, never thought about that before. I just looked it up. Apparently, at least for KiCad, the answer is nothing. Further, I also noticed in the current development branch, KiCad 5.99, cursor wrapping became optional (Wayland support is one motivation). Once "center and warp cursor on zoom" is disabled, navigation works okay. It appears the Wayland transition may make it less necessary to support cursor wrapping.

[onetuser]

Have the same issue with "geforce now" games played from browser: two mouse cursors, application cursor "jumps" brokenly, not playable.

[iacore]

~~"mouse capturing" do not exist on Linux, as applications don't need permission to read mouse movement (poor security decision of X?).~~ This is related to absolute mouse position being interpreted as relative mouse movement. #7316

Edit: There is XGrabPointer for taking exclusive control of pointer.

[iacore]

At this point we might as well support all xinput devices and copy code from some open-source remote play solution.

[maltfield]

I just opened a bug report that was marked as a duplicate of this issue

• https://github.com/QubesOS/qubes-issues/issues/8030

I'm going to add a bit of a description of the above ticket so the keywords are present in this ticket so it's easier to find for anyone else searching GitHub issues.

Specifically, the problem that I encountered was that as soon as I touch the mouse in-game, my view slips down to staring at the ground beneath my character's feet. If I move the mouse, all it does is spin around in circles while I'm stuck to stare at the floor.

Steps to reproduce

Note that this can be reproduced with a free game in apt, so anyone can test it (without having to buy, for example, Minecraft)

The below steps are specific to openarena, but this issue has been present for every FPS game that I've tried to play in Qubes.

Run the following commands to install and run Open Arena on a Debian-11-based AppVM:

sudo apt-get install openarena
openarena

Then choose the following in-game to launch into an "arena" to duplicate this bug:

1. Press <enter> to choose SINGLE PLAYER
2. Press <enter> to choose fight
3. Press <enter> to choose BRING IT ON

After a few seconds, the actual FPS game will launch. At this point, moving the mouse at all will reproduce the issue. The view on the screen will be pointed directly at the ground. Moving the mouse will just circle around the ground; you cannot look up and actually play the game.

[maltfield]

fwiw, I use FreeCAD in Qubes AppVMs and have no issue with the touchpad interacting with 3D objects in CAD.

[maltfield]

The current code simply does not support mouse capture.

I also don't understand this, because clearly when I move the mouse (trackpad) the game does respond. When I move the mouse, the view of my feet spins in a circle. If I don't touch the mouse, then the spin stops.

So mouse capture is clearly happening, it's just messed-up somehow because I can't "look up"

See also https://github.com/minetest/minetest/issues/8539

The above ticket (also due to virtualization) may provide some insight. I think I also read something about the "center" being off, causing some calibration issues with the cursor.

[DemiMarie]

@maltfield Could the problem be due to Qubes OS using core X11 input instead of XInput?

[AlxHnr]

When I move the mouse, the view of my feet spins in a circle.

This is because Qubes forwards absolute mouse positions where relative values are expected: https://github.com/QubesOS/qubes-issues/issues/7316

[Augsch123]

I think this issue is what downgrades the experience when using my organization's Solidworks through Citrix Workspace in one qube. I cannot rotate the view through middle-click drag and drop.

[e9x]

Can we get support for this soon? I've been able to work around this by passing through an entire mouse to a VM, however this solution is impractical because it doesn't let me use my mouse for anything else.

I've been able to work around the lack of hardware-accelerated graphics by upgrading to a more powerful processor. However, the lack of support for mouse capture might be a deal breaker for me using qubes, and I might switch back to an OS that's more fit for my use case.

[apparatius]

You can try this: https://forum.qubes-os.org/t/mouse-and-keyboard-passthrough-to-windows-hvm/16905/16

[e9x]

You can try this: https://forum.qubes-os.org/t/mouse-and-keyboard-passthrough-to-windows-hvm/16905/16

Interesting stuff, however I don't want to mess with xen or anything deep in my system.

However, you have inspired me to throw together a script that lets me toggle my mouse being passed through to the active VM when a key is pressed, TY!!

#!/bin/bash

# HOW IT WORKS:
# this script will redirect mouse events from DOM0 to the VM in the active window
# this should be configured to be ran whenever a keybind is pressed, this can be done via XFCE settings
# the script can be ran multiple times to toggle this

# BE SURE TO INSTALL `qubes-input-proxy-receiver` IN YOUR TEMPLATE VMS
# fedora qubes
# dnf install qubes-input-proxy-receiver

# active VM code taken from: https://github.com/QubesOS/qubes-desktop-linux-i3-settings-qubes/blob/main/qubes-i3-sensible-terminal
get_id() {
    local id=$(xprop -root _NET_ACTIVE_WINDOW)
    # had to fix this, used to be #{id##* }
    local id=${id#*window id # }
    echo ${id%%, *} # extract id
}

get_vm() {
    local id=$(get_id)
    local abc=$(xprop -id $id)
    local vm=$(xprop -id $id | grep '_QUBES_VMNAME(STRING)')
    local vm=${vm#*\"} # extract vmname
    echo ${vm%\"*} # extract vmname
}

# resources:
# https://github.com/marmarek/qubes-app-linux-input-proxy/blob/master/README.md

main() {
    # configure with your mouse
    # look for a device with the ID ending in -event-mouse
    # make sure the mouse is connected directly to dom0 and not a USB vm
    local mouse=/dev/input/by-id/usb-Logitech_G203_LIGHTSYNC_Gaming_Mouse-event-mouse

    # this script can be reworked to get the input events from a USB VM and pass them to another VM

    # lockfile path
    local tmp_file=/home/user/mouse-vm.tmp

    # get the active VM
    local vm=$(get_vm)

    # attempt to detach the mouse if we detect that iwas previously attached
    if [ -f "$tmp_file" ]; then
        local attached_vm=$(cat $tmp_file)
        #TODO:
        #kill -9 $attached_vm
        pkill -9 --full input-proxy-sender
        rm $tmp_file
        #notify-send "detached mouse from vm, killed $attached_vm"
    else
        # attach the mouse and save the state to $tmp_file
        #notify-send "active vm: $vm"
        if [[ -n "$vm" ]]; then
            #notify-send "attaching mouse to $vm"
            qvm-run -u root --pass-io --localcmd="input-proxy-sender $mouse" $vm "input-proxy-receiver --mouse"&
            # TODO: find a normal way to get the pid of the input-proxy-sender that we just spawned
            #sleep 1 # wait for the qvm-run command to spawn input-proxy-sender
            #echo "$(pidof input-proxy-sender)" > $tmp_file
            #alternatively, we can just create the $tmp_file without any contents and kill every process with the name input-proxy-senedr
            touch $tmp_file
        fi
    fi
}

main

[jamke]

Good stuff.

rm $tmp_file

This should be avoided, a better way:

rm --"${temp_file}"

[maltfield]

I've been able to work around this by passing through an entire mouse to a VM, however this solution is impractical because it doesn't let me use my mouse for anything else.

@e9x Can you please explain how you do this? Is there a way that I can pass my laptop's built-in trackpad directly to a dedicated "gaming" VM so I can finally play openareana in QubesOS? Is there an easy way to do this with the GUI?

For security reasons, I would only want to pass it to the one dedicated untrustworthy gaming VM. I would not want it to somehow accidentally be passed to my web browser VMs or -- god forbid -- my vault VMs, as some discussion above may risk.

[e9x]

Can you please explain how you do this? Is there a way that I can pass my laptop's built-in trackpad directly to a dedicated "gaming" VM so I can finally play openareana in QubesOS? Is there an easy way to do this with the GUI?

I'm not sure as I'm not able to test this on a laptop, but your trackpad should show up as a device under /dev/input/by-id/.

For security reasons, I would only want to pass it to the one dedicated untrustworthy gaming VM. I would not want it to somehow accidentally be passed to my web browser VMs or -- god forbid -- my vault VMs, as some discussion above may risk.

You can remove the VM detection code and hard code the name of your gaming VM to prevent accidentally passing it through:

    # get the active VM
    #local vm=$(get_vm)
    local vm=gpu_gaming

[e9x]

I created a gist with detailed instructions for this here: https://gist.github.com/e9x/f505cf11d42f9470570c9de0b61e15e9

[zaz]

Workaround

1. Plug in a separate USB mouse
2. Click "No" if asked if you want to use it as an input device qubes.InputMouse
3. Attach it as USB device to the desired Qube

If you're still having issues, perhaps try keeping your regular/dom0 mouse at (0,0) (should be top left) because this issue does indeed seem to be caused by #7316.

[andrewdavidwong]

Workaround

1. Plug in a separate USB mouse
2. Click "No" if asked if you want to use it as an input device qubes.InputMouse
3. Attach it as USB device to the desired Qube

Related: https://github.com/QubesOS/qubes-issues/issues/9908

[zaz]

If I understand correctly, this bug is calling to allow a VM to request to lock the cursor in-place (for whatever reason), whereas #7316 is calling to have dom0 forward mouse deltas to VMs. To manually lock the cursor in place, try

Workaround 2

sleep 5; timeout 30 sh -c 'while :; do xdotool mousemove 960 540; sleep 0.001; done'

Replace 540 960 with whatever the center of your screen is, and ensure the application is running in fullscreen mode. Remove the timeout 30 sh -c and single quotes if you're sure you can navigate back to that terminal with the keyboard only.

---

If that doesn't help the situation for you, but the application seems to work more sensibly when the mouse is in the top left corner, its likely you are affected by #7316, not mouse capture.

[maltfield]

@zaz in which terminal are we supposed to execute that command? dom0? Or the VM?

And how do we figure out what '540 960' should be on our machine? Any command to figure that out?

[zaz]

@maltfield dom0. You can use xrandr and should see something like 1920 x 1080, in which case you divide by two. Or you can search your model of screen.

By the way, I meant to say xdotool mousemove 960 540, which is for a typical 1080p screen.

[maltfield]

unfortunately that doesn't fix openarena per

• https://github.com/QubesOS/qubes-issues/issues/4986#issuecomment-1427272477

The exact same issue persists; I'm stuck starring at my feet as soon as I try to move the mouse in the game.

I tried pinning it to the center of the screen or at 0 0. The bug is still present for both.

[marmarek]

I'm stuck starring at my feet as soon as I try to move the mouse in the game.

This looks like https://github.com/QubesOS/qubes-issues/issues/7316, and there is a PR that (hopefully) will fix it.