Text lookAt camera in VR session yields incorrect positioning

[brianyuen]

Hello, I'm fairly new to three.js and currently building a fake project to help me learn. I'm looking for a bit of guidance for an issue I'm having while in "VR Mode". What I would like to do is have the text be facing the camera at all times, which seems to work while viewing the scene normally (non VR). However, when I initiate the scene in VR, the same text seems to be facing the incorrect direction as well as rotating pretty wildly as I turn my head with Oculus 2 headset on.

I'm currently leaning on the drei library to output the Text, which is wrapped with Billboard to face the camera.

Can anyone provide some guidance on why the text is facing the wrong way in VR? One idea I had was to create a simple Plane that sits just above the camera and have the Text lookAt the Plane, rather than the camera itself. Although it's just a hunch I have. Thank you!

Here is the CodeSandbox for it: https://yfdli0.csb.app/

[edwinwebb]

As you know Drei Billboard doesn't work well with VR. Happily, it's not very complicated.

Have a look at the source code (https://github.com/pmndrs/drei/blob/master/src/core/Billboard.tsx) and have it look at the Player camera you can get from React-XR.

[chillbert]

@brianyuen did you make this work? position from {player} = useXR() gives me always 0,0,0

[brianyuen]

@brianyuen did you make this work? position from {player} = useXR() gives me always 0,0,0

Unfortunately, no.

[brianyuen]

…

[bbohlender]

@brianyuen should work with v6. Context: The camera returned by useThree(state => state.camera) in v6 is now the xr camera. Therefore, all libraries consuming the camera through the R3F state, will automatically use the correct camera. Using getWorldPosition, ... on this camera will return the correct position in XR and non-XR applications.

Feel free to reopen if things continue to behave incorrectly.