Babylon.js
Babylon.js copied to clipboard
BabylonJS
WebXR improvements
- It is now possible to add a fullscreen GUI ADT to WebXR using the XR Layers feature. This reuses the same API that is used on desktop, so the experience can work on both desktop and XR. Note - no interaction is possible in XR (as this is a layer that is rendered on top of the XR environment)
- Added the ability to add a quad XR layer. At the moment only internally, as I want to change the API and make sure we can easily support all other XR layer types.
- Firefox reality is no longer a thing - removed code directly related to it.
- Added hand tracking as an optional feature - this will allow devs to use the hand meshes directly instead of seeing the static default hand models.
- Fix an issue where the baseLayer will be created twice when using XR Layers
- Added a warning when attaching or detaching a feature didn't work as expected
- Remodelled the internal architecture of WebXR layers, imrp
- Small fixes for the webxr declaration
- Moved WebXR composition layer implementation(s) to a new directory for structure. These are internal classes so it's not a breaking change.
- Fixed an issue that is apparent when using the webxr emulator
- made small changes to the session manager's architecture.
- fixed an issue with clearing when in multiview mode and in layers
Converted to draft as I want to run a few tests before merging it to be sure all scenarios work as expected.
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). To prevent this PR from going to the changelog marked it with the "skip changelog" label.
Snapshot stored with reference name: refs/pull/14753/merge
Test environment: https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14753/merge/index.html
To test a playground add it to the URL, for example:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14753/merge/index.html#WGZLGJ#4600
Links to test babylon tools with this snapshot:
https://playground.babylonjs.com/?snapshot=refs/pull/14753/merge https://sandbox.babylonjs.com/?snapshot=refs/pull/14753/merge https://gui.babylonjs.com/?snapshot=refs/pull/14753/merge https://nme.babylonjs.com/?snapshot=refs/pull/14753/merge
To test the snapshot in the playground with a playground ID add it after the snapshot query string:
https://playground.babylonjs.com/?snapshot=refs/pull/14753/merge#BCU1XR#0
Let me and @hmallen99 know when this is out of draft and we can integration test with our BabylonJS XR app on Quest 2/3, Pico 4, and Apple Vision Pro system browsers. (We can also test Wolvic on Quest 2/3 and Pico, if that's helpful for you.) We typically take up each BabylonJS release pretty quickly, but happy to test this branch specifically since it's a larger change.
Thanks a lot @matthargett !!! so cool to be able to count on the community for this !!! I bet @RaananW will be pretty happy with it as well.
Let me and @hmallen99 know when this is out of draft and we can integration test with our BabylonJS XR app on Quest 2/3, Pico 4, and Apple Vision Pro system browsers. (We can also test Wolvic on Quest 2/3 and Pico, if that's helpful for you.) We typically take up each BabylonJS release pretty quickly, but happy to test this branch specifically since it's a larger change.
Thanks a lot!
I am planning on running many different scenes using this code today and next week.
I want to test multiview and non-multiview layer sessions, the new way to add an ADT in XR, and, in general, make sure that all other scenes that are already working are working as expected. There is also the aspect of the default support for hand tracking - I am a bit afraid some devices wouldn't like it at all. I did fix a small issue that prevented this PR to work correctly on the vision OS, so knowing how other devices react to this would be great
Also important - I changed the clearing mechanism a bit. I want to know that utility scenes are working rendering correctly. So far my tests were successful.
When I start testing I will share playgrounds I am testing with.
Experimental playwright test reporter:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14753/merge/testResults/webgl2playwright/index.html
Getting started :-)
All of those scenes should work in quest OS. Layers and multiview playgrounds should fail in non-supporting devices, but everything else should work in any other device!
Fullscreen UI + Layers : https://playground.babylonjs.com/?snapshot=refs/pull/14753/merge#98tm63#228 Fullscreen UI + Layers + Multiview : https://playground.babylonjs.com/?snapshot=refs/pull/14753/merge#98tm63#227 - Found a clearing issue, working on a fix Hand support (without explicit feature enable) - https://playground.babylonjs.com/?snapshot=refs/pull/14753/merge#X7Y4H8#254 Simple scene with teleportation and input support: https://playground.babylonjs.com/?snapshot=refs/pull/14753/merge#9K3MRA#2 - This scene should work when switching from controller to hands and back to controllers.
I will continue pasting playgrounds, going to test these now.
Did some testing on Pico 4, and there's a glitch that may be the Pico's fault. I had to reboot the device to be able to run any more demos. I let Dexter (Browser maintainer on Pico) know in Twitter DM and referred him to this PR.
https://github.com/BabylonJS/Babylon.js/assets/1550766/931b40d2-dbff-4e1f-8c5f-91ddd2a49a13
Visualization tests for WebGPU (Experimental) Important - these might fail sporadically. This is an optional test.
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14753/merge/testResults/webgpuplaywright/index.html
Visualization tests for WebGPU (Experimental) Important - these might fail sporadically. This is an optional test.
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14753/merge/testResults/webgpuplaywright/index.html
I have tested these and other playgrounds and made sure they all work correctly on both the quest pro and the quest 3. I made a few changes (notably the very important dont-clear-on-your-own flag passed to the layers). Would be great to know if anyone has any other feedback!
Another playground to test with - https://playground.babylonjs.com/?snapshot=refs/pull/14753/merge#9k3mra#1430 . This one was flickering before this PR.
Did some testing on Pico 4, and there's a glitch that may be the Pico's fault. I had to reboot the device to be able to run any more demos. I let Dexter (Browser maintainer on Pico) know in Twitter DM and referred him to this PR.
Would be great to test it again and see if it was the clear flag's fault (though perfectly valid)
This is (one of) the first XR task after 7.0 is released. As you see I have not exposed the createQuadLayer so we don't have back-compat issues. I want to add the media layers first, and afterwards the rest of the shaped layers and exposing their RTTs to allow you to render anything onto them.