Rotations not working as expected? (Tracking Overrides w/ Custom Tracking)

[RaytracerDale]

trafficstars

Hi!

I'm building a driver using tracking overrides with the index as the HMD to implement a custom tracking system. I'm having trouble getting the rotations in the correct universe for what the final driver pose expects.

I've got some code that converts a rotation matrix to a quaternion (based off the Mike Day algorithm) and I'm using that same code in Unity to convert a rotation matrix to a quaternion and display it in the editor, see a example here: Video

As you can see, inside unity the green capsule is using the computed quaternion from the rotation matrix and it works just fine, but the exact same quaternion provided to the qRotation field in the DriverPose results in an inverted rotation inside SteamVR? If I flip the w of the quaternion I'm able to get a correct rotation with the camera facing forward but when facing 90 degrees it's rolling instead of pitching and facing backward it's pitching but inverted again? Here's a video of that phenomena: Weird rotations in steam vr

It's clearly still 'pitching' about a fixed axis, but as far as I can tell the maths is correct and it must be something inside the SteamVR transform stack that I'm not understanding correctly, I've tried heaps of various combinations of things to try and get a result but one axis is always flipped or incorrectly mapped or something.

I'm passing in identity quaternions for both qWorldFromDriverRotation and qDriverFromHeadRotation so this probably has something to do with the incorrect result, I must be missing something fundamental to do with the way I'm handing the universe inside the driver, does anyone have any example resources or code that could point me in the right direction.

For some additional context, my driver is based of the example in this repo.

Thanks so much in advance for any assistance :)

[okawo80085]

From the last video it looks like you're translating your rotation values to quaternions incorrectly, i had a similar issue with a math library about a year ago, it was doing quaternion operations incorrectly, so i had to do all the operations using matrices and only convert to quaternion right before i sent out the poses.

Quaternions and matrices are both used to represent transformations, but practically its waaaaaaaaaaay easier to mess up quaternion math.

[RaytracerDale]

hey thanks for the quick reply! You certainly make a great point haha, its super easy to mess up the quaternion math 😆.

At the moment I'm already using Rotation matrices right up until actually updating the pose. Here's a quick breakdown of what you see in the video:

Step 1. For each transform change I'm generating a model/TRS matrix inside of unity using the Matrix4x4.TRS function using the orange capsule’s transform.

Step 2. Send that matrix over TCP to the driver which actually performs the RotationMatrix->Quaternion conversion right before the tracker sends the pose update - this results in that weird fixed axis phenomena.

Step -2.5 Back in unity (just after sending the matrix packets), I'm using the same algorithm/code (just translated to C#) to compute the quaternion and then setting the transform.rotation of the green capsule to that newly computed quat.

Is there any reason why the math’s would work in Unity and produce the expected rotation, but inside of SteamVR it would result in that weird fixed axes thing?

[RaytracerDale]

And for context, I've tried a couple of implementations of this same operation. Here's a gist of the two main working ones.

Quaternion Implementations i've tried

For context, both of these work correctly exactly the same (as seen with the green capsule in the video). But result in the same problem inside of SteamVR.

So pretty safe to say i'm 90% sure its not the maths and something more fundamental.

[okawo80085]

Yeah looking at your math it's good, the only other thing coming to mind is maybe the order of operations maybe, but that should not matter also if you don't do quaternion math on the driver side, unless you're using relative rotations in unity (which won't work, think of drivers always providing rotation in the world global frame)

[RaytracerDale]

Sweet, good to know the math is solid, and just for the sake of clarity when you say relative rotations; are you referring to like a parent-child relationship and for example using the child’s localRotation. (Forgive me if this is a dumb question)

Because all the objects in the scene in unity are root level and have no other rotations applied. Is there some weird/wrong with how I’m making a rotation matrix from the rotation in unity? just calling Matrix4x4.TRS with the position world space rotation and vector3.one for scale. The quaternion code only uses the top 3x3 so having the full 4x4 shouldn’t matter.

I’m assuming that world space = world global frame, is that correct or are they not quite the same, again forgive me if this is an obvious dumb question 😆

This is a very peculiar issue, that you so much for your help so far by the way!

[okawo80085]

Yeah i meant world global frame and it not being the issue is more important, then you're either passing the quaternions to pose structs incorrectly or it's something else before that point, maybe on unity side.

Also, I assume you're sending your qauternions to your driver using some IPC method, make sure your quaternions are de serialized properly and don't forget that in OpenVR quaternions are written down as w x y z and:

+[0] (x) is right
+[1] (y) is up
-[2] (z) is forward

Quat strut and pose struct

[RaytracerDale]

I’m not actually sending the quaternion, I’m sending the rotation matrix and to be sure it’s getting properly interpreted on the others side I have it serialising to json at the moment, it’s definitely making its way to SteamVR correctly, the idea is to simulate the same environment as what the tracker would send from inside Unity so Unity just hijacks the same tcp server as the tracker would normally use to send a tracking message.

Right now all I’m doing is taking that matrix packet deserialising it and running the exact same code on as on the unity side which is that QuaternionFromMatrix function and then piping the result of that into a quaternion like this: vr::HmdQuaternion_t poseQuat = {quat.w, quat.x, quat.y, quat.z};

is there anything else that could cause this behaviour?

[okawo80085]

Ok now im concerned about the math again... but also your pose struct, can you show your entire pose struct that gets used as the device's pose?

And about what else could be causing this, the nature of quaternions mostly, you see it's hard to notice if you messed up in quaternion transformations because unless you go through your entire range of motion you might miss it.

Gotta warn ya though, even if turns out all of my previous concerns are null, there is a chance you're getting bad matrices from unity, which are fine for unity, but don't work well with OpenVR, that can be checked too though. Just replace your unity provided matrices with matrices that you define by hand and check if the orientation of the quaternions aligns with what you set in your matrix, if not boom thats your answer, your conversion math is messing up somewhere.

[RaytracerDale]

hi, hope you had a good easter!

Here's my full driver pose:

vr::DriverPose_t outPose = { 0 };

outPose.deviceIsConnected = true;
outPose.poseIsValid = true;
outPose.result = vr::ETrackingResult::TrackingResult_Running_OK;
outPose.willDriftInYaw = false;
outPose.shouldApplyHeadModel = false;
outPose.qDriverFromHeadRotation = { 1.0, 0.0, 0.0, 0.0 };
outPose.qWorldFromDriverRotation = { 1.0, 0.0, 0.0, 0.0 };

vr::HmdQuaternion_t quat = QuaternionFromMatrix(rotationMatrix);
outPose.qRotation = quat;

outPose.vecPosition[0] = 0;
outPose.vecPosition[1] = 1.89f;
outPose.vecPosition[2] = 0;

What orientations do you think I should manually try? And is there any specific way for how to generate these matrices, the way I know is this way:

And is there any special format to be weary of with the quaternions or matrices?

Here's another video with a bit more info as well: https://www.youtube.com/watch?v=AKa48tk4JtM

[okawo80085]

Sorry for the late reply, your pose struct looks good

What orientations do you think I should manually try? And is there any specific way for how to generate these matrices, the way I know is this way

You should start with every axis individually first, then start combining them with matrix multiplication

And is there any special format to be weary of with the quaternions or matrices?

There was a really good issue in pyrr's repo about that, i'll link it if i find it again.

Here's another video with a bit more info as well

Yeah so inverted axis is normal and it looks like your unity coordinate system is rotated 180 degrees around the y axis, that'd cause pitch and yaw to be inverted at the same time, apply a 180 rotation around the y axis to your matrix using matrix multiplication before you convert it quaternion.

[Flutterish]

Hey, maybe this wont really help, but I had to flip both w and z to get the rotation to work correctly. You said you only flipped w.