XRLayer stereo layout not supported in the fallback

[geyang]

The following example does not work as intended as far as I know:

  const video = useMemo(() => {
    const result = document.createElement('video');
    result.src = 'https://threejs.org/examples/textures/MaryOculus.webm';
    result.crossOrigin = 'anonymous';
    return result;
  }, []);
  return   
              <XRLayer
                position={[0, 0, -0.5]}
                onClick={() => video.play()}
                scale={0.5}
                src={video}
                // shape={'cylinder'}
                layout={"stereo-left-right"}
              />

shows up like this:

[bbohlender]

@geyang yes, the non-xr fallback does not support stereo content yet

[geyang]

Yeah I just tracked it down in the source code.

[geyang]

It looks like even in the vision pro, under XR, the stereo content does not work

[geyang]

It looks like this XRLayer API is unstable. Will switch to custom implementation.

[edhyah]

@geyang Looks like I'm always a week behind what you're working on haha. Did you switch to using an imperative implementation via vanilla three.js?

[geyang]

Hey @edhyah so Vuer actually had this since Dec 2023 😝 , and then WebRTC support August 2024.

My impression is that the XRLayer support across vendors is currently inconsistent. On the Vision Pro, this library actually uses regular quads as a fallback. So, after tracing the code, I decided to use my original quad instead.

[edhyah]

@bbohlender I think I found the issue, at least on my end with a Quest 3.

In createXRLayer, createXRVideoLayer or createXRNormalLayer are called, both of which call getSpaceFromAncestors which returns state.originReferenceSpace. However, in the start of the session, the state.originReferenceSpace is not populated and is null (it is populated later). Because it's not populated, createXRVideoLayerand createXRNormalLayer returns undefined.

The fix that works for me is to have the line state.originReferenceSpace = xrManager?.getReferenceSpace() ?? null; before the return statement of createXRLayer, like so:

export function createXRLayer(src, state, xrManager, relativeTo, options, properties) {
    state.originReferenceSpace = xrManager?.getReferenceSpace() ?? null;    // Fix
    return src instanceof HTMLVideoElement
        ? createXRVideoLayer(src, state, relativeTo, options, properties)
        : createXRNormalLayer(src, state, xrManager, relativeTo, options, properties);
}

Is that a valid fix? If so I can create a PR. This fix makes the example code work for me.

[geyang]

This is awesome!

[bbohlender]

@edhyah @geyang I fixed the XRLayer issue. Thanks for your investigation @edhyah

I'll now move onto fixing the camera layer problem, which will then enable fixing this problem :)