[MRTK3] Using OculusXR feature in the OpenXR Feature Group causes quest controller visuals to be glitched

[akash14darshan]

Describe the bug

When I enable the "OculusXR Feature" in OpenXR Feature Groups (to enable Quest specific things like Passthrough, OVRInput, etc) the controller gets visual gets bugged, and the Far interaction ray looks bent. The controller model seems to be stuck in the air (as seen in screenshot) The controller does seem to respond ingame though to rotations. But movement seems to not work.

Hand tracking seems to work fine without any issues.

To reproduce

Steps to reproduce the behavior:

1. Configure the project as mentioned in the Microsoft's guide
2. Goto ProjectSettings/XR Plugin Management/Open XR, and select android operating system.
3. Enable "OculusXR Feature" in OpenXR Feature Groups.
4. Build and deploy to device

Expected behavior

Normally I would expect this feature not to interfere with the controllers.

Screenshots

Your setup (please complete the following information)

Unity 2021.3.8f1 MRTK3 Repo (as of commit https://github.com/microsoft/MixedRealityToolkit-Unity/commit/87ae775470952d28f0d459e9abeb6b95102505ff)

Target platform (please complete the following information)

• Quest 2

Additional context

I'm not sure if its helpful or not, but OVRPlugin.GetSystemHeadsetType() returns OVRPlugin.SystemHeadset.Oculus_Quest instead of OVRPlugin.SystemHeadset.Oculus_Quest_2 on Quest 2

[Zee2]

@keveleigh , are we supporting this path at all?

[keveleigh]

Oh hmm...I haven't done much with Oculus' feature here since it's a little heavy 😅 but it seems weird to me that it'd affect our own input! I can see about testing this today

[akash14darshan]

I dont know if this info could be useful or not, but from as what I can think of, it seems like the Ray is always coded to originate from the controller model prefab.

When I added OVRManager prefab (with controllers) in the script with "Disable Eye Anchor Cameras" I noticed that hand tracking was still fine and unaffected, but with controller mode, I could see the Quest's original controller model (of Quest1 instead of Quest2) on screen which was moving and following my physical controllers movement.

Other than that, rest all was the same. The MRTK controller was still stuck and the ray behavior was same aswell.

[mikeskydev]

I haven't been able to get Oculus's OpenXR feature to work properly at all, it seems very early still. Is this really an MRTK bug?

[akash14darshan]

I haven't been able to get Oculus's OpenXR feature to work properly at all, it seems very early still. Is this really an MRTK bug?

It seems so because MRTK Inputs should not be affected by enabling Oculus XR OpenXR feature.

[Zee2]

This is sort of tricky because we're not explicitly supporting using OVR APIs or OVRManager with MRTK3. However, as it stands, things probably shouldn't be interfering.

[akash14darshan]

I did some research on it and i think the root cause is the rotationAction and the positionAction in the action based controller. They do work normally but for reason with Oculus XR enabled, it doesn't work.

I did find a workaround for it. There are two requirements for it:

• Detect if hand tracking or controller I'm doing this using Unity Input System's Oculus Touch Left/Right controllet presence.
• Create new Input Asset which will use Oculus Touch Controller Left and Right device position and rotation respectively.

Now create a new script, attach it to Mrtk XR rig prefab. i) when the controller mode is detected, get reference to left and right action based controller. save references of old position and rotation action (to restore later). Then apply the new position and rotation action which we created from Oculus Touch Controller. ii) when hand tracking is back, restore back the old position and rotation action.

I don't know if I my steps were clear or not, but using this workaround I am now able to use controllers in Oculus xr + mrtkv3 normally.

PS: I will gladly respond to anything if required, but i still hope this workaround becomes a part of core mrtk itself since now I'm able to use all the Oculus XR specific features with MRTK3 which is a huge plus for me. It would be a bonus for many other quest developers who would like to use quest specific features like Passthrough, etc.

[Zee2]

@akash14darshan thanks for the extra info here. There should be a way we can get around this without doing funky runtime input action swapping... Pinging @keveleigh here :)

[keveleigh]

Confirmed repro over Link 😡 will dig in to see why this feature is causing issues!

[keveleigh]

Hey @akash14darshan! I was previously able to repro, I believe with the 46.0 version of Meta's Oculus Integration package. After updating to 47.0, I can no longer repro. Are you able to update and give this another shot without your workaround?