godot_openxr icon indicating copy to clipboard operation
godot_openxr copied to clipboard

Have OpenXR create a depth texture swapchain

Open BastiaanOlij opened this issue 3 years ago • 8 comments

This PR adds support for creating a depth texture swapchain and supplies the correct depth texture to Godot so Godot uses it to render the depth buffer into.

This depth texture is then supplied to OpenXR. OpenXR can use this for advanced reprojection and for tools like LIV.

This needs thorough testing, on face value it seems to work but as it's a big structural change, best to make sure this works on all platforms first.

BastiaanOlij avatar Mar 01 '22 05:03 BastiaanOlij

Ok, with @ChristophHaag fixes added in it looks like its now submitting the depth buffer properly but for me in SteamVR the re-projection screws up, it's like it's trying to reproject into the wrong direction

BastiaanOlij avatar Mar 02 '22 01:03 BastiaanOlij

Even though this pretty much works and is ready to merge I've changed this back to a draft PR for two reasons:

  1. At the moment this is broken on SteamVR, I've added a test for that to disable it but we may want to postpone merging this until we are sure this is indeed the issue and Valve lets us know its fixed
  2. I think the way we change our framebuffer in Godot core is causing a performance hit, I'm going to work on a PR to fix that

BastiaanOlij avatar Mar 03 '22 09:03 BastiaanOlij

Reminder to myself, need to check XR_KHR_COMPOSITION_LAYER_DEPTH_EXTENSION_NAME and only add the depth layers if it's supported.

BastiaanOlij avatar Mar 03 '22 23:03 BastiaanOlij

Reminder to myself, need to check XR_KHR_COMPOSITION_LAYER_DEPTH_EXTENSION_NAME and only add the depth layers if it's supported.

@m4gr3d @ChristophHaag I'm now checking if this extension is available and too my surprise it's not available on Quest? Might only be available on Quest 2? It is available on the Oculus desktop runtime.

BastiaanOlij avatar Mar 05 '22 08:03 BastiaanOlij

I've put this back to ready for testing, it now properly checks for the extension and I think the code in the plugin itself is fine. Further testing on the performance hit, I think my computer just needed a restart or it may have been the buggy code in SteamVR itself. I can't find any noticable performance difference running with and without the depth buffer logic on when running on the Oculus Runtime.

BastiaanOlij avatar Mar 05 '22 08:03 BastiaanOlij

Reminder to myself, need to check XR_KHR_COMPOSITION_LAYER_DEPTH_EXTENSION_NAME and only add the depth layers if it's supported.

@m4gr3d @ChristophHaag I'm now checking if this extension is available and too my surprise it's not available on Quest? Might only be available on Quest 2? It is available on the Oculus desktop runtime.

@BastiaanOlij I don't think the Quest (Oculus mobile runtime) supports depth based reprojection. The closest equivalent would be spacewarp which has its own XR_FB_space_warp extension.

m4gr3d avatar Mar 07 '22 16:03 m4gr3d

@m4gr3d in due time we'll need to make it so that we have this solution active once we fully support spacewarp though that will likely be a Godot 4 feature as there are no plans that I am aware off of adding motion vectors to Godot 3.

Bit of a shame that they're not sticking to the standard though, I get that they need their own extension to submit the motion vector buffer but it would make sense if they had implemented submitting the depth buffer according to the existing API.

Alas, standards are made to be broken I guess :)

BastiaanOlij avatar Mar 08 '22 12:03 BastiaanOlij

Ok, I think I've taken this as far as I can for now, we could merge it in just so all the code is there.

For SteamVR we'll remove the check once I get confirmation the issue on their end is resolved. Valve don't use the depth buffer themselves but they do send it on to the Oculus runtime, so it is strange it stops their platform from working properly.

Once we have something for motion vectors we can add spacewarp support (again possibly not until Godot 4) and tie into the depth buffer code here.

BastiaanOlij avatar Mar 10 '22 02:03 BastiaanOlij