Volumetrics-io
60 FPS
This is a general issue for tracking our frame rate target.
current FPS in headset is a consistent 30fps.
Confirmed the performance problem. On my non-gaming laptop + Chrome, the demo top page (localhost:8080) runs at about 30fps.
I commented out Three.js Renderer.render() and then the fps became about 60fps. So perhaps this performance problem may be from rendering stuffs, not ECS, Physics, or other systems running on CPU.
Shadowing is the first suspicious to me. The fps became 60 on that top page if I commented out the shadow initialization.
The current shadow map size 2k may be a bit too huge for the performance. The fps became 60 if I changed the size to 0.5k (the Three.js default).
We may need to seek a good balance between the quality and balance, or other workarounds or shadow implementation if we need the both.
Perhaps configuring the shadow parameter that is tradeoff between performance and quality won't truly satisfy us. It may take time to completely resolve the shadow performance and quality issues so we may need to think of short-term workaround for now because that performance problem may give a bad impression and experience to users. Potential (temporary) solution ideas
- Configurable shadow map size (use smaller one if quality is not required)
- Use smaller shadow map size in non immersive devices
Related to rendering performance, Three.js doesn't seem to support multiview extension but a PR is opened https://github.com/mrdoob/three.js/pull/25981 I'm not really sure when and whether Three.js will support it. If we need it, we might may a Three.js fork and apply a patch.
Related to rendering performance, Three.js doesn't seem to support multiview extension but a PR is opened mrdoob/three.js#25981 I'm not really sure when and whether Three.js will support it. If we need it, we might may a Three.js fork and apply a patch.
I've considered this for other features as well. Depth sensing just landed a few weeks ago but there's no to set the depth distance.
My one concern is diverging too much. What are your thoughts?
My one concern is diverging too much. What are your thoughts?
I really don't like having our own fork because of high maintenance cost. But we might be able to think of it in the following cases
- If official Three.js unlikely supports a feature soon and custom change has a big performance or quality impact. When we consider this we should first test and compare the difference ideally in quantitatively and then decide to go with it. Ideally we need to come up with a low maintenance cost patch.
- If official Three.js likely support a feature soon but more early adoption has a huge market impact. In this case we might not need to think of maintenance cost because we may dispose the fork when official Three.js supports that feature.
We should have the courage not to fork unless the impact is clearly significant.
This aligns with my own thoughts. I'm open to exploring a fork, but only once we've optimized every where else.
We could also try to upstream the changes from our fork? making sure to only change one feature at a time.
I left a comment on the PR https://github.com/mrdoob/three.js/pull/25981#issuecomment-1958530030
It appears that EtheralEngine and AFrame came to the same conclusion. I'm more open to it if the performance gains are significant, but I'd much rather push for it to make it into actual THREE.js if we can.
https://github.com/EtherealEngine/etherealengine/pull/9056
https://github.com/supermedium/three.js/pull/15
only once we've optimized every where else.
Yes, agreed. I have an impression of that performance bottleneck is somewhere else.
We could also try to upstream the changes from our fork? making sure to only change one feature at a time.
Might be less trouble in upgrading (merging) if we always apply the patches on top of the official Three.js?
It appears that EtheralEngine and AFrame came to the same conclusion
Picking a multiview extension commit from there would be the easiest solution if possible.
I'd much rather push for it to make it into actual THREE.js if we can.
Exactly
The comments for the performance above are from my non-gaming laptop Windows + Chrome. I'm also testing the performance in immersive mode on Quest 3 + Meta Browser.
On Quest 3 + immersive mode + Meta browser, what I found so far are
- Runs at about 45fps
- FPS number became near to 90 fps if disabling the both masking system and shadows
- Disabling either shadows or masking system has no impact to fps number
I will investigate more...
Testing environment
- Meta Quest 3 + Meta browser
- Skybox example
- Just opened the example page and enter immersive mode. No other interaction
- Hacked the stats panel a bit (use stats.update() and show the stats panel even without debug mode)
Regarding the shadow map in Three.js, I found that WebGL texture is created with specified width * 4 and height * 2 for PointLight, for example if pointLight.shadow.mapSize.set(2048, 2048) then WebGL texture size will be 8k x 4k. It sounds like pretty huge. If acceptable, using different light for shadow can speed up the app by reducing the GPU memory usage.
If acceptable, using different light for shadow can speed up the app by reducing the GPU memory usage.
This is a good idea. Didn't realize how big the map would be.
The one thing you'll need to do is figure out how to beat position the light. We could try attaching a spotlight to the camera, ensuring shadows would be cast on whatever is in front of the user. We can also play with the position so the light is always coming from above.
Opened a separated issue for shadow map optimization for better tracking of the discussion #498 because there may be no easy solution.
I'm currently using Quest 3 as a performance measurement device to achieve 60 (90, or more higher) FPS. Recently, we have tried various optimizations on MRjs, but we have not been able to reach 60 FPS stably yet. Therefore, we have come to want to evaluate the performance of Quest 3 itself.
If we cannot reach 60 FPS due to the hardware limitations of Quest 3 (and not due to the inefficiency of MRjs), there is little point in trying to improve the efficiency of MRjs. Instead, we will need to take a different approach, such as improving performance with a trade-off in quality.
Also, if native VR/AR app performance is good on Quest 3 while WebXR app isn't we may need to give feedback more aggressively to Quest team (or major device team).
From @michaelthatsit on discord
reduced bind rendering example: https://immersive-web.github.io/webxr-samples/reduced-bind-rendering.html source: https://github.com/immersive-web/webxr-samples/blob/main/reduced-bind-rendering.html
framebuffer scaling example: https://immersive-web.github.io/webxr-samples/framebuffer-scaling.html source: https://github.com/immersive-web/webxr-samples/blob/main/framebuffer-scaling.html
Especially framebuffer scaling is one of the performance optimizations I had in my mind if we allow performance/quality tradeoff.
I'm more interested in testing multiview performance recently because I started to more think that rendering stuffs are one of the biggest performance bottlenecks.
If we cannot reach 60 FPS due to the hardware limitations of Quest 3 (and not due to the inefficiency of MRjs), there is little point in trying to improve the efficiency of MRjs. Instead, we will need to take a different approach, such as improving performance with a trade-off in quality.
I'm quite certain the hardware isn't the bottleneck.
If you try this example, you'll see that it reaches 90 FPS on Quest 3.
https://immersive-web.github.io/webxr-samples/immersive-ar-session.html
keeping in mind that it's a much simpler scene. There are also a few THREE.js examples that are more complex, these might be better benchmarks.
With #516 and #522, the skybox example in immersive mode seems to start to run nearly at 90 fps on my Quest 3. I'm happy if you test on your end, too.
There is a chance that Element.getBoundingClientRect() (or reflow caused by it) can drop the FPS number significantly, refer to #493
We may close this issue because we reached target frame rate with the recent updates?