neuralnet tracker pitching upwards on yaw

Hello,

I've been using the neuralnet tracker for several months now, and while it has generally performed quite well, I have been encountering a positioning issue that I can't quite resolve. Basically, when I turn my head left or right (yaw) the head tracker pitches up. Even after calibrating and centering the tracker, this issue persists.

Just for context, my camera is sitting on top of my monitor, and angled downwards towards my head, so I'm not looking directly at the camera, but rather, slightly below it.

I've fiddled with the settings quite a bit, but have yet to find a suitable resolution for this problem. Did I miss a setting that would resolve this issue?

Hello,

I've checked on my side, I get the same kind of behavior between pitch and yaw (although I don't use yaw angles as wide as you do), but I noticed that this behavior was the same with previous version 2022.3.2 (with the settings I used for that version)

I did not care too much about it, assuming it was due to how the webcam was seeing my face, and then deducing the 6 DOF parameters

But maybe there's something to check here with the ONNX lib outputs?

Try changing the centering method:

Options - Shortcuts - Centering method - Point

@GO63-samara Thanks for the suggestion, but I already currently use Point as the centering method

It is quite possible this is the behavior of the neural-net model which I haven't been able to get rid off.

Here is a plot from a recording of myself. Pitch vs Yaw. I tried to keep pitching to a minimum and maybe failed at it.

Here is a similar plot for some snippets of the BIWI dataset. It contains different subjects with ground truth poses under lab conditions. Different color means different snippet.

You can see there are "wavey" deflections from the expected pitch. The noise-filtering in opentrack would hardly get rid of that.

Thank you for the response @DaWelter

In my case, I believe this issue is rooted in the fact that my face is not directly aligned with the camera, but rather slightly offset below the camera (i.e. my camera is sitting on my monitor angled down towards my face).

I assume moving my camera so it is below me angled upwards would have the opposite effect: the tracker would pitch downwards on yaw.

Would it be possible to allow the end user to dampen the pitch at the extremes of yaw based on the offset angle of their camera? Example, by default the offset is set to 0. Positive values dampen negative pitch. Negative values dampen positive pitch.

Thoughts?

And how about the Wireless VR 360 option, @ber8749 ?

Judging from the code & math behind it, I would expect it to be the best option to compensate for the camera offset. You look straight at the middle of the monitor and hit the center button.

I recall there was a "camera pitch" setting in earlier versions of opentrack. Not sure how exactly it worked though.

@sthalik , maybe we need a fixed center-pose based on proper rotations (in SO3 space). Implemented with quaternions it'd look like (Quat(zero_pose_euler_angles).inverse() * Quat(head_angles)).as_euler(). The current "custom center pose" implementation doesn't do the trick, I believe, because the values are subtracted axis by axis in Euler angles.

I think you know, but still, here is an illustration of what I mean Imagine adding 30 deg pitch on both sides as "centering" correction. The left panel would be fine. But the pose on the right panel will have wrong pitch as well as roll. If otoh we measure the right pose w.r.t the pose on the left, we see it's a rotation around the green axis, hence only yaw rotation.

@DaWelter That sounds like a much more elegant solution. Great idea!

So, @ber8749 , does this "Wireless VR 360" option help?

Sorry, @DaWelter, I misunderstood your earlier comment.

Unfortunately, the method you describe (look at middle of monitor and hit "center" button) does not dampen the unwanted pitch when turning my head.

Just to be 100% clear ... Is set, right? Then I'm out of ideas.

Oh okay, I didn't realize "Wireless VR 360" was a centering option, my mistake. So, I just tried using it, and unfortunately it did not seem to dampen the unwanted pitch at the extremes of yaw.

Regardless of outcome, thank you for looking into this.

Just wanted to mention that it is still the problem. It doesn't matter if neuralnet or other method (tracker marker, for example) is used, so it seems it is, indeed, problem with opentrack centering calculation.

I'm wondering if anyone did find a solution/ workaround? I'm having the same issue.

I'm having the same issue, everything I have tried nothing seems to be working. Has anyone found a solution for it?

Sadly, the only thing that I can suggest to somehow mitigate the issue, is to place camera above the screen/head plane, not below (if possible). The issue still exist in both cases, but is much more pronounced if the camera is looking up at you.

I hope proper fix will get integrated at some point :(