Automatically calibrate cursor position to match head tilt (Bonus: generalizes head tilt mode)

[1j01]

Currently, the user's face is detected in order to place tracking points, and optical flow is then used to track these points on your face, and only the optical flow influences the final cursor movement.

In this existing scheme:

• Cursor movement is directly affected by the camera's perspective.
• The head's tilt is not used directly.
• Translating your head has as much of an affect as tilting it (or more).
• When the cursor runs into the edge of the screen, it stops, whereas your head can continue of course, and this naturally leads to a way of "calibrating" on the fly to a more comfortable range of head movement.
• However, you have to think about it a bit.

What if we auto-calibrated based on the head tracker's face orientation, perhaps making adjustments only during otherwise-detected movement? Like a perpetual motion machine's secret magnetic "kick", or a magician's slight of hand, it would subtly adjust the mouse position so that it ends up at the edges of the screen when tilted a certain amount, and centered when facing forward.

Drawbacks:

• It may not actually feel subtle in practice, when strong enough to work, or may feel subtly wrong and degrade the experience overall in a way that's hard to put your finger on, since it's like a magic trick.
• Tilt is relative to camera position! So this auto-calibration would need its own calibration.
• This may just be overcomplicating things.

Formula: Assuming the tilt can be normalized, or assuming the camera and screen and face are directly in line with each other, a formula for the other part might be fairly simple, something like adjusted_x = x + (abs(delta_x) * auto_calibrate_strength * (x_implied_by_tilt - x) where x_implied_by_tilt is the x position on the screen that would be mapped purely from the head tilt where delta_x = x - x_previous where x is the latest x position from optical flow tracking It's probably a little more complicated, like maybe the delta_x factor should be raised to some power, or clamped, etc.

On the plus side, it could be made so that with auto_calibrate_strength = 1, it purely uses the head tilt, so a separate head tilt mode wouldn't be needed. (Again, I'm not sure about the delta_x part in regard to this.)

[1j01]

Maybe it should only make adjustments in the direction that you're already moving, i.e. if sign(delta_x) = sign(x_implied_by_tilt - x) I think that would help with the sleight of hand, although I'm still not sure if sleight of hand is desirable.

If nothing else, it would be an interesting experiment!