servo-unity icon indicating copy to clipboard operation
servo-unity copied to clipboard

OES_EGL_image_external?

Open jdm opened this issue 4 years ago • 3 comments

From https://news.ycombinator.com/item?id=23953778:

  | jayd16 13 hours ago  [–]                                                >As we do not have  control over the binding of the texture and the Unity context, the  current design for updating this texture uses a blit (copy) via Servo’s  surfman-chains API

I don't think this is necessary.  You should be  able to make Texture objects that ref existing native textures in Unity  and I've been able to use OES_EGL_image_external extensions to sample  Android decoded video textures sampled from Unity shaders.Its not the most widely available option but the perf is much better.

jdm avatar Jul 26 '20 15:07 jdm

You should be able to make Texture objects that ref existing native textures in Unity

👉 https://docs.unity3d.com/ScriptReference/Texture2D.CreateExternalTexture.html

(The other way round: https://docs.unity3d.com/ScriptReference/Texture.GetNativeTexturePtr.html)

bddckr avatar Jul 26 '20 19:07 bddckr

The issue is not access to an external texture, it's access to an external texture on the right thread in the right GL context. We did look at writing a new backend for surfman to integrate directly, but it's a bit of work and not at all cross-platform, so this was not a goal of the project at this stage. I'd like to find a way to avoid the blit though, so I'm continuing to experiment in this direction. (Same response posted in the HN thread).

philip-lamb avatar Jul 27 '20 00:07 philip-lamb

You should be able to make Texture objects that ref existing native textures in Unity

(The other way round: https://docs.unity3d.com/ScriptReference/Texture.GetNativeTexturePtr.html)

This is how we're doing it at the moment. https://github.com/MozillaReality/servo-unity/blob/dc55c92f3b3e57dbec04614db0f21d79ffbfa032/src/ServoUnity/Assets/Scripts/ServoUnityWindow.cs#L207

I think the OP on the HN thread is referring to Unity's ability to wrap a Texture2D object around an existing GL texture created outside Unity. I've used this approach in the past, but the challenge with doing that here is that we'd need to add a new backend to surfman to bind a texture from the Unity context on the Unity rendering thread as the RTT target for surfman. It's not trivial.

philip-lamb avatar Jul 27 '20 00:07 philip-lamb