Asynchronously allocate XRWebGLLayer's GPU-side resources

[MortimerGoro]

The WebXR spec is using promises in many places but not for the XRWebGLLayer creation:

• Allocate and initialize resources compatible with session’s XR device, including GPU accessible memory buffers, as required to support the compositing of layer.
• If layer’s resources were unable to be created for any reason, throw an OperationError and abort these steps.

Because of this UA's might need to add a blocking ipc/queue call in the JS process if GPU resources are allocated in another process or queue. Would it be worth it to add an async API for XRWebGLLayer creation? (this could be applied to WebXR Layers too). Or any way to report the OperationError later?

[Manishearth]

We could paper over this with context loss. Unsure. Servo managed to make this work without blocking calls -- I think we allocated a handle and had the rest of the operations happen off thread.

[Manishearth]

A thing I'll point out: none of these steps must happen until it's fed to the XRSession via updateRenderState, and that does happen off thread.

[cabanier]

A similar issue was filed against the layers spec: https://github.com/immersive-web/layers/issues/252

[toji]

I think this is a decent thing to evaluate for the layers repo, as Rik linked above, but I don't have much motivation to update the core spec for this. The core XRWebGLLayer is known to have flaws and I would like to treat it as effectively deprecated once we can get broader layers support shipped. As such I think we should focus our efforts on improving the API surface there.

As for the synchronous nature of the API: Returning handles for GPU resources happens "synchronously" according to the WebGL API, but functionally I think all implementing browsers treat it as an asynchronous request under the hood. For what it's worth WebGPU is going to follow the same pattern: Creation of resources is a synchronous operation, but the thing that's returned is actually just an opaque handle that actually gets backed by GPU memory sometime later. Given that browsers have been able to handle this pattern fine thus far I'm not sure what the benefits would be to having this API specifically be more asynchronous.

[MortimerGoro]

I agree that we should focus on layers and that it's not the time for breaking changes in the core API. I wanted to implement the opaque pattern for this. The problem is the spec states that the OperationError happens in the constructor instead of handled later.

The idea that @Manishearth and @cabanier mentioned of handling the check in updateRenderState could work.

Another way to handle this would be to use the Opaque Framebuffer completion status for this scenario. If the allocation failed, the FBO would be incomplete.

[Manishearth]

We should probably have some text in the core text talking about this:

As for the synchronous nature of the API: Returning handles for GPU resources happens "synchronously" according to the WebGL API, but functionally I think all implementing browsers treat it as an asynchronous request under the hood.

As well as further specifying where opaque framebuffer errors occur.