Cursor position off in full screen mode (only for “full width” aspect ratio)

[jotaen4tinypilot]

Description

When “Full Screen” view is turned on, the cursor position is off in case the aspect ratio is “full width” (i.e., the remote screen occupies all available horizontal space, while there is padding at the top and the bottom). In this case, the cursor misaligns in vertical direction.

There is no problem if the aspect ratio is “full height”.

Just for clarification what is meant by “full width” and “full height”:

Full width: ←max→

Full height: ↑max↓

What's the behavior that you expect?

Correct and reliable alignment of local and remote mouse cursor position during full screen mode.

Investigation

The cause is that the <remote-screen> component doesn’t correctly recognise that the remote screen should be full width. Instead, it erroneously reports the screen to still be in full height mode.

This screenshot was taken while the remote screen was actually in full width mode:

When manually changing the attribute from fullscreen-is-full-height to fullscreen-is-full-width in that situation, the problem is gone. So the attributes themselves do correctly what they should, it’s only the detection/toggling mechanism that seems to be flawed.

So it appears that the problem is related to how we detect “full height” and “full width” mode, and how we toggle between these two states and both respective attributes.

A starting point for the fix might be the fillSpace method in the <remote-screen> component.

What are the steps to reproduce this behavior?

I can observe different behaviour in Chrome and Firefox.

In Chrome, when entering full screen mode, the initial state always appears to be correct. Only when the aspect ratio changes during full screen mode, the issue appears.

In Firefox, the initial state appears to be broken right away after having entered full screen mode.

For debugging/reproducing, I opened the browser’s dev console, and then attached it either to the bottom, or to the side, in order to influence the aspect ratio during full screen mode. (Since in full screen mode, you obviously cannot resize the window otherwise.)

Screenshots

Screen recording taken during full screen mode, with “full width” aspect ratio. The vertical cursor alignment is off. It’s okay around the vertical center line of the screen, and gets worse the more you move the cursor to the top or bottom border.

https://github.com/tiny-pilot/tinypilot/assets/83721279/3cd41785-3385-4a65-b277-e9e05874ea04

[jotaen4tinypilot]

The issue also seems to be a bit random/erratic, see https://github.com/tiny-pilot/tinypilot/pull/1583#pullrequestreview-1591862867.

Maybe the resize event handler doesn’t fire reliably or so?