godot_openxr_vendors icon indicating copy to clipboard operation
godot_openxr_vendors copied to clipboard

Add Android Surface swapchain support

Open maunvz opened this issue 1 year ago • 4 comments

This is a draft for adding an AndroidSurfaceLayer object that extends Node3D and does the following:

  • Creates an Android Surface backed swapchain
  • Sets up an XrCompositionLayerQuad to submit (using Node3D pose, visibility, etc)
  • Stores an in handle for that Surface that can be passed to an Android plugin to retrieve the Surface

With this change, a project can use Android Surfaces to display content on OpenXR layers and control them with the Godot scene graph.

TODO: Add an example to the demo project

maunvz avatar Mar 24 '24 05:03 maunvz

We'll need to see how this overlaps with the logic we're building to core around composition layers. Also seeing these are all KHR extensions it probably belongs in core, not in the vendors plugin.

BastiaanOlij avatar Mar 25 '24 04:03 BastiaanOlij

I already have a nearly finished PR for Godot that adds support for composition layers, including XrCompositionLayerQuad. I'll see if I can finish it up quick, or maybe just make a draft PR..

dsnopek avatar Mar 25 '24 11:03 dsnopek

Here's my composition layer PR: https://github.com/godotengine/godot/pull/89880

dsnopek avatar Mar 25 '24 13:03 dsnopek

I spent some more time looking into this!

If I'm understanding correctly, the XR_KHR_android_surface_swapchain extension allows creating a swap chain where the data drawn onto the swapchain is pulled from an Android Surface. So, if you've got an Android plugin that's drawing to an Android Surface, you could use this extension to display that on an OpenXR composition layer.

In that case, I think it would probably make sense to support this by building on PR https://github.com/godotengine/godot/pull/89880, to allow configuring the composition layer nodes to use an Android Surface rather than a Godot SubViewport as the source for the composition layer.

And, I agree with @BastiaanOlij that this should be done in Godot (rather than 'godot_openxr_vendors') because the core extension is XR_KHR_android_surface_swapchain, which is vendor neutral.

The PR here also uses XR_FB_android_surface_swapchain (a Meta vendor extension), but if we want to keep that bit out of Godot, we'll need to add some hook to Godot that allows GDExtensions to modify the swap chain creation. But that shouldn't be a big change, and we'll likely already need something similar to allow modifying composition layer creation (to support numerous other vendor extensions, like, for example, XR_FB_composition_layer_secure_content).

dsnopek avatar Apr 02 '24 18:04 dsnopek

I just posted PR https://github.com/godotengine/godot/pull/96185 which adds support for XR_KHR_android_surface_swapchain into Godot itself.

It doesn't include support for XR_FB_android_surface_swapchain_create since that's a vendor extension. In order to add that via GDExtension we'll need to make some other small Godot changes, but that'll come in a future PR.

Closing this PR as superseded by https://github.com/godotengine/godot/pull/96185

dsnopek avatar Aug 27 '24 21:08 dsnopek