OffscreenCanvas in a Worker support for webxr

[ranbuch]

WorkerNavigator doesn't support xr property.

[Manishearth]

I believe this is intentional (see https://github.com/immersive-web/webxr/issues/345), but I'm not 100% sure, @toji can you confirm?

[toji]

Intentional in the sense that we earmarked it as an additive feature to explore after the core feature set launched. To my knowledge there's no reason WebXR couldn't run in a worker and in fact it may be the most performant place to run your session. Biggest pitfalls would be synchronization of video or audio content, and any interaction with mouse, keyboard, and touch inputs. which to date are still tied to the main thread.

Mostly we just need to decide when we want to put the time into doing a rigorous review of what it will take to add worker support.

[ranbuch]

@toji those are great news!

With regarding to the main thread synchronization issue, you are right, but it's already the case for OffscreenCanvas.

It's a pain but it's manageable and the performance boost is totally worth it.

Here is a comparison between main-thread rendering VS OffscreenCanvas rendering: The middle one is the OffscreenCanvas and the FPS is somewhere around 40% faster.

With regards to input synchronization here is a live example.

About XR, using Scene Viewer is awesome if you want a general experience. But if you need a custom experience you are force using your own solution (open with Chrome for Android). Unfortunately, when using WebXR, we don't have the performance boost, arguably when we need it the most.

BTW, models doesn't look the same when moving from one viewer to another so it's best serving it all on the same environment.

Obviously, we still have issue with IOS. It's probably here to stay for the next year (at least).

I really hope that I managed to maybe promote the development of this feature.

Thanks!

[ranbuch]

@toji Hi there,

Apologize for being a bit pushy but do you have an ETA for this feature?

Are you working on it or do you just acknowledge the need and won't prioritize it in the near future?

Thanks.

[Manishearth]

There is no ETA, we're not prioritizing this

[ranbuch]

Hi @Manishearth,

Can you please reconsider prioritising this issue? WebXR experience on low-end devices is really choppy and we can also do better for all devices.

Thanks.

[cabanier]

At least for the Oculus platform, workers wouldn't result in faster performance. HMDs with mobile chipsets already have a high load from the regular 2D browser threads and world recognition.

Workers will compete for resources with these processes which will result in lower and uneven performance. It's better to optimize your scene to fit in the frame budget or to offload content that is not sensitive to timing to workers.

[ranbuch]

Thanks for taking the time to answer @cabanier.

I guess that might be the case on mobile devices if that didn't work for Oculus. Be that as it may, rendering the scene from another thread has many benefits (as I have demonstrated above and as Google has demonstrated) but I cannot use it if there's a chance the user might invoke the augmented reality experience.

That means that by not supporting WebXR on WorkerNavigator you are assentionaly deprecating the OffscreenCanvas option for many use-cases.

Kindly reconsider.

[Manishearth]

Ultimately, prioritization is tied to interest from browser vendors in actually making a change, so it's not really up to us to "reconsider" here. We aren't determining that this issue is not worth solving; just that we're not able to spend any time on this right now.

This is far from a simple issue to solve.

[cabanier]

That means that by not supporting WebXR on WorkerNavigator you are assentionaly deprecating the OffscreenCanvas option for many use-cases.

You can continue to use workers and offscreen canvas for content that is not performance sensitive. Check your experience and see how your scene viewer could benefit from that.

I'm surprised to see such a difference in performance between main thread rendering and using canvas workers. Why would a worker be able to render faster? Can you share the sites you posted above?

[ranbuch]

You can continue to use workers and offscreen canvas for content that is not performance sensitive. Check your experience and see how your scene viewer could benefit from that.

Scene Viewer is not open source as far as I know and doesn't use WebXR, it's a native Android application. In case you referred to model-viewer, what they are doing is lowering the resolution of the canvas. The FPS stays reasonably high but the quality significantly decreases.

Really liked the hack for creating more than 15 webgl instances by creating only a single context and rendering the rest on a 2d canvas but obviously you are going to get that for free with an OffscreenCanvas.

The image that I've posted above is a screenshot from a Laptop with 8th gen Intel U CPU with 16GB. I was getting 60 FPS on all viewers but not simultaneously (while opening 3 tabs at once).

BTW, Google Poly has added OffscreenCanvas support but there's no longer access for the platform. The viewer is still alive as far as I know and Google is using it to display 3D ads (I might be wrong about that!).

I've created a demo where you can see how OffscreenCanvas is better enduring the multiple scenes. In case you are getting the max FPS just simulate stress by clicking the jank button.

Here is a screenshot of this demo of the same mentioned Laptop:

Is using a different thread always better? no, not always. but taking this option away isn't going to hurt only my use-case but many more, that's for sure. Probably yours as well someday.

If one wishes to empower the web platform, support more use-cases and provide better support for the users I believe the choice is obvious. Unfortuneltly, I have no say in the metter, the power is in your hands, the big companies. We, the community, can only sound our voice with hope that it will be heard.

Thanks.

[ranbuch]

@Manishearth Just noticed your last comment. That's unfortunate but I can relate to that. Thanks for your comment.

Obviously I'm clueless but is it possible to pass the XR session to the worker via post message or stream the session data via BroadcastChannel?

Maybe there's a relatively simple workaround?

[AdaRoseCannon]

Looking at https://immersive-web.github.io/webxr-samples/tests/offscreen-canvas.html

Makes me think this is supposed to work now right? If so I think this issue can be closed.

[toji]

Unfortunately not quite. We do support OffscreenCanvas on the main thread. We don't yet support WebXR on Worker threads, which is the primary request of this issue from my reading.