three.js icon indicating copy to clipboard operation
three.js copied to clipboard

Webxr cpu

Open MichaelLangbein opened this issue 6 months ago • 1 comments

It seems that, as for now, threejs doesn't support cpu-optimized WebXR depth-sensing.

I'm using an older device (a Pixel 3) and found that a lot of the WebXR examples don't work on this phone. I think there are two issues with supporting this device (and probably a few other order devices, too), but please do correct my if I've misunderstood anything:

  1. Currently, threejs only instantiates glBinding = new XRWebGLBinding if session.renderState.layers exists, that is, if the WebXR-layers module is being supported (code here).
  2. My device can only handle cpu-optimized depth sensing; but currently threejs seems to only use the gpu-optimized access method (code here, cpu-style access method described here)

So I've been playing around with this a little bit - I hope that this PR can serve as a basis for discussion.

I guess we'd have to figure out a few questions, though:

  1. Does threejs want to support cpu-optimized depth-sensing?
  2. If so, one reasonable way to go about this would be to create a cpu-equivalent of the current WebXRDepthSensing class - this PR creates a WebXRDepthSensingCpu class.
  3. Since the cpu based depth-data is not provided as a WebGL texture but as an array on the cpu, I don't think one could mirror the WebXRDepthSensing API surface perfectly: for example, we could create a WebGL texture from the cpu-array to mirror the getDepthTexture method, but that would involve a lot of copying of data from cpu to gpu.
  4. So in the most general form: what should a possible WebXRDepthSensingCpu look like?

With or without cpu-optimized depth-sensing, threejs has pretty nice WebXR support - thanks btw for all those examples! For now I've refrained from creating a concrete example for cpu-based depth-sensing, just because I guess that from this discussion some significant changes might still come about.

Cheers!

MichaelLangbein avatar Aug 04 '24 09:08 MichaelLangbein