[Blocker] Users expect constraining ObjectManipulator's position will result in a lever-type interaction

[Zee2]

Describe the bug

When users set ObjectManipulator to only set rotation, not position, they expect a "lever"-type interaction, where the object pivots around its center, dragged by the manipulation point.

The current math does not implement a lever-type interaction. Selecting only rotation results in the same type of direct manipulation, but with the position component suppressed. This results in confusing behavior where the object seems to rotate opposite from the "dragging direction". This is because the interaction being computed is not a pivot or lever interaction, but simply a direct interaction that has one of its components suppressed.

The fix

Implement an alternative lever-type manipulation logic, depending on whether position or rotation is permitted.

Additional context

Matt R will fill in additional context on DRIs, blockers, etc.

[holomatt]

DRI: Chad Pfarr

Observed: Behavior does not work like MRTK v2 when it comes to setting up ObjectManipulator for Rotation Only. Rotate around Grab point and Rotate around Object center options also appear to have no effect in this configuration.

Expected: User will intuitively move and rotate their hand when rotating a movement constrained object. For 6dof gesture this should collectively take into account the hand movement in addition to rotation as it is not intuitive to hold your hand completely in place and twist the wrist. Ideally, this configuration behaves similar to / the same as MRTK v2.

This issue blocks inclusion of ObjectManipulation interactions into the experience when configured as described due to the interaction being non-intuitive to users.

[Zee2]

This is relevant to @RogPodge's rewrite of constraints/solvers systems, the above behavior is the result of a common method for apply constraints in order.

[Zee2]

For a workaround for anyone reading here: you can use this PivotInteractable to model the lever-like behavior folks described above.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR.Interaction.Toolkit;
using Microsoft.MixedReality.Toolkit;

/// <summary>
/// Models a lever-like pivot interaction around a specified axis.
/// </summary>
public class PivotInteractable : MRTKBaseInteractable
{
    // Will constrain pivoting to the specified axis.
    // e.g., mode = X => only spin around X axis.
    public enum ConstrainMode
    {
        Free,
        X,
        Y,
        Z
    }

    [field: SerializeField, Tooltip("Which axis to constrain pivoting to. (e.g., Y results in only spinning around Y axis")]
    public ConstrainMode AxisConstraintMode { get; set; } = ConstrainMode.Free;

    [field: SerializeField, Tooltip("If set, the script manipulates this transform instead of the transform it's attached to.")]
    public Transform CustomTargetTransform { get; set; } = null;

    [field: SerializeField, Tooltip("The custom pivot point to use.")]
    public Transform CustomPivot { get; set; } = null;

    // Which pivot are we actually using?
    private Transform pivot => CustomPivot != null ? CustomPivot : target;

    // Which target are we actually using?
    private Transform target => CustomTargetTransform != null ? CustomTargetTransform : transform;

    // The vector from the pivot to the interaction point at the start of the interaction.
    private Vector3 fromVec;
    
    // The rotation of the target at the start of the interaction.
    private Quaternion fromRot;

    // The vector from the pivot to the interaction point at the start of the interaction.
    private Vector3 pivotOffset;

    protected override void Awake()
    {
        // We're not poke-able.
        DisableInteractorType(typeof(IPokeInteractor));
    }

    protected override void OnSelectEntered(SelectEnterEventArgs args)
    {
        base.OnSelectEntered(args);
        RecordFromVector();
    }

    public override void ProcessInteractable(XRInteractionUpdateOrder.UpdatePhase updatePhase)
    {
        base.ProcessInteractable(updatePhase);

        if (updatePhase == XRInteractionUpdateOrder.UpdatePhase.Dynamic && isSelected)
        {
            Vector3 toVec = ProjectToConstraintPlane(interactorsSelecting[0].GetAttachTransform(this).position - pivot.position);

            // If we've rotated far enough, let's reset our fromVec to avoid
            // wrap-around-the-sphere error.
            if (Vector3.Angle(fromVec, toVec) > 45)
            {
                RecordFromVector();
            }

            Quaternion rot = Quaternion.FromToRotation(fromVec, toVec);
            
            // If pivot == transform, then this pivot offset math is no-op.
            target.position = Vector3.Lerp(target.position, rot * (pivotOffset) + pivot.position, 0.5f);

            // Simply apply the rot to the fromRot. We'll periodically reset
            // the fromRot to avoid wrapping error.
            target.rotation = Quaternion.Slerp(target.rotation, rot * fromRot, 0.5f);
        }
    }

    private void RecordFromVector()
    {
        fromVec = ProjectToConstraintPlane(interactorsSelecting[0].GetAttachTransform(this).position - pivot.position);
        fromRot = target.rotation;
        pivotOffset = target.position - pivot.position;
    }

    private Vector3 ProjectToConstraintPlane(Vector3 vec)
    {
        switch (AxisConstraintMode)
        {
            case ConstrainMode.Free: return vec;
            case ConstrainMode.X:    return new Vector3(0, vec.y, vec.z);
            case ConstrainMode.Y:    return new Vector3(vec.x, 0, vec.z);
            case ConstrainMode.Z:    return new Vector3(vec.x, vec.y, 0);
            default:                 return vec;
        }
    }
}