Camera positions and webxr offsets

[danrossi]

I've tried reaching out elsewhere with no success. I think these are bugs. For the non webxr camera and orbit controls the equi example defaults to the camera looking to the left. In WebXR it flips it 180 degrees so looks the other way. I have hacks to change the camera position so it looks to the front but WebXR needs an offset to not look to the side using the camera positions. It should be using the camera position and rotation as a base reference by default.

Attempting webxr to follow the camera position and rotation in an offset kind of works but doesn't. Its not looking straight ahead and certain y rotations and x positions makes the view rotated sideways.

Is there a fix for equi video to make the camera look straight ahead not default to the side and get WebXR to follow the initial positions of the non webxr camera.

https://dev.electroteque.org.s3.amazonaws.com/three/examples/webgl_video_panorama_equirectangular4.html

[danrossi]

This code almost gets the camera offset looking to the front not the headset default of 180 degrees to the left or right. With the requirement of an 0.5 value for some reason. It gets crazy from there, my external test, the X offset works but trying to integrate it doesnt do anything. There is a requirement to position the x rotation so when tilting up its the camera default position, required for sitting flat in a chair. So sitting back doesnt rotate to the sky

    const xrManager = this.renderer.xr,
    camera = this.camera,
    baseReferenceSpace = xrManager.getReferenceSpace(),
    offsetPosition = camera.position,
    offsetRotation = camera.rotation,
    //transform = new XRRigidTransform( offsetPosition, offsetRotation  );  
    transform = new XRRigidTransform( offsetPosition, { x: -offsetRotation.x, y: -(offsetRotation.y + pos - 0.5) , z: offsetRotation.z, w: offsetRotation.w } );  

    const teleportSpaceOffset = baseReferenceSpace.getOffsetReferenceSpace( transform );

    xrManager.setReferenceSpace( teleportSpaceOffset );
    ```

[danrossi]

It's really fiddle to get the default camera positioned to the front right, I have to update its x position. It seems I have a better offset working now to match the original camera position and rotation so its not defaulting 180 degrees. If you orbit up and down it will match it in the headset. The same code wont work in my integration yet. Fiddling with mesh rotations doesnt help.

https://dev.electroteque.org.s3.amazonaws.com/three/examples/webgl_video_panorama_equirectangular5.html

[shi-lai-mu]

I also encountered a similar problem. When simulating movement, the X offset is always greater than 0, resulting in reverse movement. The same is true for the official example "teleport". At present, it is not clear how to solve this problem?

[danrossi]

Took ages but my offset code works. The final code looks like this when launching a session. Orbit controls needs to be updated manually once to get the final camera rotation. A fraction of 0.5 is required to get perfect centre. Make sure headset is calibrated first also.

I had a specific requirement to replicate the x rotation of the non webxr camera for using a headset lying down in a chair. So I have that as a config.

Thanks for adding the reference space method to begin with.

this.currentControls.update();
     
        const xrManager = this.renderer.xr,
        camera = this.camera,
        baseReferenceSpace = xrManager.getReferenceSpace(),
        offsetPosition = camera.position,
        offsetRotation = camera.rotation;

        const transform = new XRRigidTransform( offsetPosition, { x: this.config.xrTiltOffset ? offsetRotation.x : 0, y: -(offsetRotation.y - 0.5) , z: offsetRotation.z, w: offsetRotation.w } ),
        teleportSpaceOffset = baseReferenceSpace.getOffsetReferenceSpace( transform );

        xrManager.setReferenceSpace( teleportSpaceOffset );
       ```

[shi-lai-mu]

@danrossi Thank you very much for the reference, we seem to have different problems, but this code led me to the cause of the problem! :)

[danrossi]

I've got extra tests to display the situation. Non-WebXR camera will display at a wrong position unless the x position is corrected, or the mesh y is rotated and camera z position is set to -1. If I rotate the mesh, it needs to be inverted in webxr because webxr inverts everything, but the reference offset cannot be set or it breaks.

In the corrected example setting the camera x position so it displays at the front. The WebXR camera needs the reference offset set to the non webxr camera or it displays to the right of the front position. An extra offset of 0.85 is needs to get exact centre and this value became different for different videos.

I need the reference offset to match the rotation and position of the non webxr camera so it can start when the axis is tilting up for when sitting down in a chair !

Starts facing the ground, in webxr it displays 90 degrees to the right of the front.

https://dev.electroteque.org.s3.amazonaws.com/three/examples/webgl_video_panorama_notcorrected.html

Camera x position set to half of PI displays to the front. WebXR needs an extra offset value of 0.85 to display to the exact centre. I need to figure out how to avoid correcting the camera and not needing the 0.85 value

https://dev.electroteque.org.s3.amazonaws.com/three/examples/webgl_video_panorama_corrected.html

Using mesh rotate and z position set. faces the front in both non webxr and webxr but offsetting breaks it and the position is at the ground.

https://dev.electroteque.org.s3.amazonaws.com/three/examples/webgl_video_panorama_meshrotate.html

With webxr offset set it displays at the ground

https://dev.electroteque.org.s3.amazonaws.com/three/examples/webgl_video_panorama_meshrotate_offset.html

[danrossi]

Is there a simple fix for this ? It seems when I try and apply the x rotation to tilt up is not directly centre and neither is the y rotation. The x rotation is needed for when lying down with a headset on

[danrossi]

It was suggested to change to quaternion and it seems to work better and centred properly. I'll close this and use this as a guide.

renderer.xr.addEventListener('sessionstart', (e) => {

			controls.update();

            const baseReferenceSpace = renderer.xr.getReferenceSpace();
            
            const offsetPosition = camera.position;

		
			const offsetRotation = camera.quaternion;

            const transform = new XRRigidTransform( offsetPosition, { x: offsetRotation.x, y: -(offsetRotation.y), z: offsetRotation.z, w: offsetRotation.w } ); 

            const teleportSpaceOffset = baseReferenceSpace.getOffsetReferenceSpace( transform );

            renderer.xr.setReferenceSpace( teleportSpaceOffset );

		});