ValveSoftware
GetEyeToHeadTransform not working (wiki <-> doc inconsistency)
I can't get GetEyeToHeadTransform to work. It seems left and right matrices are flipped (I need to use the right matrix for the left image and vice versa). The main problem is that the Wiki documentation and the openvr.h documentation are contradicting each other. I don't know which one is true and can't figure it out.
Wiki Documentation states: "Instead of Model * View * Projection the model is Model * View * Eye * Projection" openvr.h documentation states: "Instead of Model * View * Projection the sequence is Model * View * Eye^-1 * Projection"
According to my tests the left matrix returned by GetEyeToHeadTransform has "-0.0303" which indicates openvr.h to be right. But then the 3d in the hmd totally breaks down. Using the wiki documentation seems to fix the 3d breaking down (at last it feels better) but then you have to flip the matrices around.
What is true?
NOTE: I assume the "left eye" is the eye as seen from the point-of-view of the user, not looking at the user from the outside. This is not documented and is thus also unclear (and has catastrophic consequences) but could be a potential explanation for left/right being flipped.
The header is the correct one.
If you look at the example application, the eye matrix is inverted in the function that calls GetEyeToHeadTransform: https://github.com/ValveSoftware/openvr/blob/master/samples/hellovr_dx12/hellovr_dx12_main.cpp#L1811
The order-of-operations for calculating the view-projection matrix are reversed though. https://github.com/ValveSoftware/openvr/blob/master/samples/hellovr_dx12/hellovr_dx12_main.cpp#L1831
Figured out the problem. The projection matrix values returned are for right-handed system but I'm using left-handed. I needed to invert the m[0][2] and m[1][2] values in the constructed projection matrix to make it work. So the flipped values in the projection matrix caused the eye images to drift apart depending on distance.