UTM icon indicating copy to clipboard operation
UTM copied to clipboard

Please adapt to iPadOS 16 Stage manager and external display

Open lurenJBD opened this issue 11 months ago • 7 comments

Describe the issue need to get UTM's mouse capture improved for iPadOS 16.

Configuration (required)

  • UTM Version: 4.4.5
  • OS Version: iPadOS 16.3.1
  • Device Model: iPad Pro 2021 11 inches
  • Is it jailbroken (name jailbreak used)? No jailbreak,using TrollStore2
  • How did you install UTM? TrollStore2

Crash log

No Crash.

Debug log

maybe later.

Upload VM

I am running Windows 11 23H2 aarch64 in UTM

Here is a demo video of the bug

First bug: Abnormal display of virtual machine contents in UTM.

This bug also exists on iPadOS 15 when using AirPlay for casting. Maybe it's similar to this issue #5724

Temporary workaround: open UTM in the iPad Pro and run the VM, then transfer it to the external screen.

https://github.com/utmapp/UTM/assets/31967654/418a86e8-e219-44ac-a78f-e0bea204d49a

Second bug: If you run UTM alone, the mouse cannot be captured under external screen. Or maybe it's locking the mouse in the UTM VM, and you can't get it out.

https://github.com/utmapp/UTM/assets/31967654/2a223072-860e-4fa5-bb0a-745554e25fc5

Third bug: running two apps in the external screen, UTM can capture mouse movement, but it will be shifted. But you can resize the app window, and UTM will adaptively adjust the VM resolution as well

https://github.com/utmapp/UTM/assets/31967654/2f7dd266-7b3b-4623-8d45-f7fffc131527

lurenJBD avatar Feb 27 '24 06:02 lurenJBD

Regarding the UTM casting via AirPlay display exception, it doesn't seem to exist under UTM 4.3.5. In this issues mention the improved #4725

https://github.com/utmapp/UTM/assets/31967654/373a2d61-efd2-4825-87f8-ee3e13ca41d0

lurenJBD avatar Feb 27 '24 06:02 lurenJBD

Additions, in Stage manager scheduling Non-full screen running UTM captures the mouse and does not release it, requesting a shortcut key for the action

https://github.com/utmapp/UTM/assets/31967654/2c73f0c3-03be-44bb-9722-37cb704decbd

lurenJBD avatar Feb 27 '24 07:02 lurenJBD

Once 4.5.x is out please try it and report back because the auto resolution feature was redesigned.

osy avatar Feb 27 '24 07:02 osy

Once 4.5.x is out please try it and report back because the auto resolution feature was redesigned.

Ok, I'll test it after the UTM update.

lurenJBD avatar Feb 27 '24 07:02 lurenJBD

Improvements to the cursor capture function

@osy I've found that Microsoft's RD Client handles cursor capture in iPadOS 16's external display quite well.

Can both accurately capture the cursor and freely move out of the app

Better than I originally envisioned, no need to set up a special shortcut for releasing the captured cursor

https://github.com/utmapp/UTM/assets/31967654/bed3c50f-a8df-45c4-8423-629364b275cb

About using the Picture-in-Picture function

Then I had a new idea, why not use picture-in-picture to show the resource usage of the current VM, isn't that what QEMU Guest Agent is designed for?

It is even possible to print the QMP output log in a picture-in-picture.

I'm just asking if this feature was considered, not trying to have it in the 4.5 update

Questions about AirPlay

Then I had a question about AirPlay, when I launch UTM in my iPad Pro, the cast display is replaced with a paragraph containing the text "Waiting for VM to connect to display".

Is it possible to have an on/off switch for this auto screen casting feature? Because it would replace the external screen desktop mode of iPadOS 16

lurenJBD avatar Feb 27 '24 13:02 lurenJBD

About 4.5.0 Beta Status Report

1. Display exception bug

Starting a VM with UTM on an external screen doesn't cause the previous display exception anymore

https://github.com/utmapp/UTM/assets/31967654/42c90025-dfde-407e-b7a7-abc954519d6a

However, the second screen via AirPlay casting still shows the same exception as in version 4.4.x.

https://github.com/utmapp/UTM/assets/31967654/98be736d-812b-43f4-9658-be90ed14506f

2. Cursor Capture Bugs

4.5.0 Beta is unchanged from the previous version and still doesn't work well on external screens.

Add some descriptions that I missed before:

UTM toolbar animation on external display is not working correctly, and the functionality of the toolbar is not working correctly (it works before VM startup, but after VM startup, clicking on it doesn't work).

Using the split-screen button (three dots) at the top of the app to enter full-screen mode, the VM resolution occasionally doesn't auto-adapt, and the cursor doesn't shift position, but it can't be moved out of the app either, it's locked into the UTM.

https://github.com/utmapp/UTM/assets/31967654/b8fd340c-def0-4d22-a0a1-058ff59e2be5

lurenJBD avatar Feb 28 '24 08:02 lurenJBD

Can confirm. I'm on 16.3.1 and 11" m1 user; 16gb ram. Having the same issues. This setup is very close to being able to replace my macbook, just would need this external display full screen to work, cause 11" is unsustainable for longer periods

20bhavik01 avatar May 14 '24 12:05 20bhavik01