Migeran LibGodot Feature

[kisg]

Features

• Compile Godot Engine as either static or shared library.
• Expose Godot Engine controls (startup, iteration, shutdown) over the GDExtension API to a host process.
• On Apple Platforms, Godot Engine windows (both the main window, and other windows created after startup) may be rendered into surfaces provided by the host process.
  • This support can be extended to other platforms, read on for details.
• Sample applications (separate repository)
  • C++ Sample (tested on MacOS and Linux)
  • SwiftUI Sample (tested on iOS)

Why LibGodot?

LibGodot has a number of different use cases.

• Control Godot Engine from a host application
  • E.g. Start Godot from a .NET process to use standard .NET tooling
• Use it for automation of development tasks (e.g. building an asset pipeline using a Python GDExtension API)
• Embed Godot into another application while displaying Godot Windows as part of the host application's UI.

How to try it out?

We provide a Github repository with preconfigured build environment, sample apps and more information: https://github.com/migeran/libgodot_project

Migeran LibGodot Design

The core idea of the LibGodot feature is to build upon Godot's strong extension capabilities: its modularity, powerful type system and the GDExtension API.

The main advantage of this approach is, that this way LibGodot may be used from any language that has a GDExtension binding (e.g. C++, Swift, Rust, Python, ... etc.), with only minimal changes to the binding.

Godot Instance Lifecycle Management

The main class added by the LibGodot design is the GodotInstance:

class GodotInstance : public Object {
	GDCLASS(GodotInstance, Object);

	static void _bind_methods();

	bool started = false;

public:
	GodotInstance();
	~GodotInstance();

	bool start();
	bool is_started();
	bool iteration();
	void shutdown();
};

This class is made accessible over the GDExtension API. This class can be used to control a Godot instance.

To actually create a Godot instance a new symbol is added to the GDExtension API:

GDExtensionObjectPtr gdextension_create_godot_instance(int p_argc, char *p_argv[], GDExtensionInitializationFunction p_init_func);

This function can be used to create a new Godot instance and return a GodotInstance object reference to control it. Both samples show how easy it is to bind this function and then use the generated GDExtension API bindings with the returned GodotInstance object.

To properly destroy a Godot instance the GDExtension API is extended by another symbol:

typedef void (*GDExtensionInterfaceDestroyGodotInstance)(GDExtensionObjectPtr p_godot_instance);

This function is made available through the GDExtension API's getProcAddress mechanism.

Note: Due to Godot's internal architecture (the use of global data structures and singletons) only one Godot instance may be created in a process.

Embedding Godot UI

Note: UI embedding is currently implemented for Apple platforms. Please read the Next Steps section for more information on the status of other platforms.

To allow for embedding the Godot UI into a host process the host process needs to be able to pass the necessary data about a native surface where Godot may render its contents. The form of this native surface is entirely platform and rendering backend specific.

The Migeran LibGodot design adds the following types to Godot to allow this:

• A new DisplayServerEmbedded implementation which uses externally provided native surfaces as its rendering targets.
• A RenderingNativeSurface class and its associated platform specific subclasses, e.g. RenderingNativeSurfaceApple.
• The Window class is extended by a set_native_surface(Ref<RenderingNativeSurface>) method which allows specifying the rendering target of a Window in a typesafe, platform independent manner. It is even possible to change the rendering target of a Window dynamically during its lifecycle.

These classes are all exposed to the GDExtension API, so it is easy to use them from the host process.

The DisplayServerEmbedded class also exposes methods that allow the injection of input events from the host process into the Godot instance.

The RenderingNativeSurface class hierarchy has an additional function: provides a mechanism using the Factory Method design pattern to create compatible RenderingContextDrivers for a native surface.

This allowed us to make the DisplayServerEmbedded class platform agnostic.

We also refactored the other DisplayServer implementations to use RenderingNativeSurface during initialization.

Since all these classes are exposed over the GDExtension API, these can be seamlessly used from any supported language with a GDExtension binding.

Rationale for RenderingNativeSurface Design

For those who are familiar with Godot Engine internals: there was already a way to pass in platform specific native surface data (called WindowPlatformData) during initialization.

Why is this refactoring necessary to a full fledged reference counted object?

• We chose reference counting because it makes it easier to use on the client side, no need to manage the lifecycle manually. Earlier versions of this PR used simple Godot Objects, but they required manual memory management which was error prone.

• While on Apple platforms it is very easy to pass in a CAMetalLayer reference, and Metal (and thus MoltenVk) will happily render into it, other platforms impose way more restrictions.

  • For example: On Windows and Linux a separate Vulkan instance and the use of external textures is required to render Godot on a separate thread from the host application's main renderer thread.

  • This is not just a theoretical option: We already implemented a prototype on Linux and Windows based on Godot 4.2, where the host process and Godot are using Vulkan External Textures to pass the rendered frames from Godot to the host process. We intend to upgrade and refactor this patch to the current version of the LibGodot patch.

  • To use External Textures a thread-safe queue has to be implemented between between the host process and Godot, and a reference-counted RenderingNativeSurface subclass would be an ideal place to implement it, e.g. RenderingNativeSurfaceVulkanExternalTextureWin32.

Next Steps

Open for Discussion

We are happy to discuss any part of the design, naming conventions, additional features ... etc.

Merging for Godot 4.3 as an Experimental Feature

• We propose to merge the current LibGodot patch as an experimental feature in Godot 4.3, because it is already very usable in its current form for many use cases.

Godot 4.4 Developments

During the Godot 4.4 development cycle additional features may be added, for Example:

• Support for additional platforms for UI embedding: Android, Windows, Linux (both X11 and Wayland).
• Support for OpenGL (Native / ANGLE) UI embedding on all platforms.
• Add a Configuration API that can be used on startup instead of supplying command line parameters.

Sponsors & Acknowledgements

• Initial development sponsored by Smirk Software
• Rebasing to Godot 4.3 and further development sponsored by Xibbon Inc.

* The GDExtension registration of the host process & build system changes were based on @Faolan-Rad's LibGodot PR: https://github.com/godotengine/godot/pull/72883

About the Authors

This feature was implemented by Migeran - Godot Engine & Robotics Experts.

• Production edit: This closes https://github.com/godotengine/godot-proposals/issues/6267.

[fire]

You can fix some of the github ci actions tests with pre-commit run -a to format the text.

Here are the rest of the errors. https://github.com/V-Sekai/godot/actions/runs/8642913827

[fire]

I've tried to get the Godot Engine Github actions continuous integration to pass. Most tests pass except for Windows. https://github.com/V-Sekai/godot/tree/vsk-libgodot-migeran-4.3

Edited: Time is limited for me to work on this, so I hope this can help your efforts.

[kisg]

Thanks! I'm so excited to finally see this PR :-)

Thank you, and thank you for taking the time to review it!

I looked only at the GDExtension-related changes so far. There's a bunch of rendering and display server related changes that I haven't looked at.

At high-level, this seems like a pretty good approach. Also, I really appreciate that the code changes to GDExtension are fairly minimal. :-)

On PR #72883, we discussed the possibility of introducing a new concept, perhaps called GDExtensionLoader, that would encapsulate the loading behavior, so that we could have GDExtensionLibraryLoader that used OS::open_dynamic_library() (so the current way) and then maybe a GDExtensionEmbeddedLoader that would just take an initialization function.

However, given how simple the changes are here, I think we could perhaps save something like that for a future refactor? It could make things a little cleaner by not having to constantly check if GDExtension::library is null or not, though.

I think introducing the concept of library loaders would benefit not just this use case, but also other language support that are implemented as GDExtensions. If you recall, I proposed a feature like that in my .NET integration proposal: https://docs.google.com/document/d/1QwZpo3oIKHyita1mDOIB_xIxdwJ5rtAmpY4_vP9S02Y/edit?usp=sharing

However, I am also not sure if we should try to push this whole loader concept into this PR, we could do that in a follow-up PR. We can discuss this more in the GDExtension meeting.

[kisg]

I've tried to get the Godot Engine Github actions continuous integration to pass. Most tests pass except for Windows. https://github.com/V-Sekai/godot/tree/vsk-libgodot-migeran-4.3

Edited: Time is limited for me to work on this, so I hope this can help your efforts.

Thank you @fire ! Would it be ok if I squash your changes into a single commit, and add it to our branch?

[fire]

Do what what works for you.

Edited:

As the initiator of the libgodot-style patch, I've successfully persuaded several individuals, including @Faolan-Rad to contribute to the libgodot project. I am excited about the new places we can use Godot Engine.

• Elixir-02 branch on V-Sekai's Godot repository
• Godot Engine Proposal #4773
• Blog post on Tirania.org

[fire]

@kisg Can you review the changes to libgodot from here https://github.com/V-Sekai/godot/pull/35/?

Here's the modified libgodot_project project working on Windows. https://github.com/V-Sekai/libgodot_project. You may need to force mingw.

[GeorgeS2019]

@RandomShaper

Are you familiar with the DirectX12 surface so the UI is not just a desktop BUt standard userControl compatible with e.g. Avalonial or WPF or WinForm? https://github.com/migeran/libgodot_project/commit/246cb91cd8a4255f605653869a912d6076a5efa7

@fire How do you create a window desktop application with multiple subwindows

WPF DirectX Extensions:

Microsoft offers a library called WPF DirectX Extensions that allows you to host DirectX 10 and DirectX 11 content within WPF applications. With this library, you can create a D3D11Image control in your XAML, which acts as a surface for rendering DirectX content. Unfortunately, WPF DirectX Extensions currently only supports Direct3D 11, but the principles are similar for Direct3D 12. You can find the library on GitHub and use NuGet packages to get started1.

Custom Solutions:

For Direct3D 12, there isn’t a direct native way to host it inside a WPF window. However, you can explore custom solutions by creating your own D3D12Image control (similar to D3D11Image). This approach involves more low-level work, as you’ll need to manage the interaction between WPF and D3D12 surfaces.

[fire]

For the purposes of this libgodot pull request I am postponing these for the future as listed in the start:

• Android support?
• WPF (w/ WPF DirectX 12 Extensions)
• Support for OpenGL (Native / ANGLE) UI embedding on all platforms.
• Add a Configuration API that can be used on startup instead of supplying command line parameters.

There are several unfinished review comments from @dsnopek to be worked on.

TODO:

• [ ] library loaders integration
• [ ] Pick one consistent name for libgodot
• [ ] Godot Engine was not designed for repeated initialization although there is a way to rename all symbols with a prefix or suffix it is not for now.
• [ ] Register or not register the GodotInstance class, so that it can't be created or used in ways we don't want

[kisg]

Thank you all for your reviews and contributions. We are working on some updates and will include the contributed features as well. Stay tuned!

[octanejohn]

does this help with running/embedding the game in editor related https://github.com/godotengine/godot-proposals/issues/7213 https://github.com/godotengine/godot-proposals/issues/9142

[dsnopek]

@octanejohn:

does this help with running/embedding the game in editor

I don't think so. This would allow other applications to embed Godot, but I don't think it would allow Godot to embed Godot. Those proposals offer different approaches to accomplish their goals

[fpdotmonkey]

This would allow other applications to embed Godot, but I don't think it would allow Godot to embed Godot.

Genuine question, what's the difference?

[kisg]

With this latest update most review comments should be fixed. New implementation uses GDExtensionLoader PR as the base: https://github.com/godotengine/godot/pull/91166

Also updated to a recent master.

Still on the TODO list:

• Windows build
• Fix CI builds
• Create a .NET/C# example using the new GDExtension-based .NET implementation.

[GeorgeS2019]

Create a .NET/C# example using the new GDExtension-based .NET implementation.

See https://github.com/godotengine/godot/pull/72883#issuecomment-2226296833

[AThousandShips]

@YakoYakoYokuYoku Where did you leave suggestions? I can't see any code suggestions

[GeorgeS2019]

@AThousandShips @kisg The windows build PR is ready for merge I think this PR is ready to merge too. Please feedback

[Update] 24.07.2024 => now that we have

• go
• c++
• swift
• Next is c#

to create sample app https://github.com/JiepengTan/libgodot_llgo_demo/issues/1

[akien-mga]

@GeorgeS2019 We're in feature freeze for 4.3, and this PR is slated for 4.4. Please be patient.

[YakoYakoYokuYoku]

@YakoYakoYokuYoku Where did you leave suggestions? I can't see any code suggestions

I've messed up with the PR review, my bad. Regardless, here they go.

[rovantiq]

this PR is slated for 4.4

@akien-mga Is the plan to ship 4.4 with the unified editor (assuming things go well with this and related PRs)?

[kisg]

this PR is slated for 4.4

@akien-mga Is the plan to ship 4.4 with the unified editor (assuming things go well with this and related PRs)?

@rovantiq Could you please point me to the unified editor PRs / proposals that you are referring to?

[Gnumaru]

@rovantiq Could you please point me to the unified editor PRs / proposals that you are referring to?

Maybe rovantiq was refering to the possibility of related PRs, not to specific PRs that he could remember. I remember myself this topic of having an unified .net editor but I can't remember where I first read about it. I have just searched and found a mention about a .net contributors meeting here https://github.com/godotengine/godot-proposals/issues/7895#issuecomment-1737839064, I'm just copy-pasting the text here for convenience.

Since I don't see it mentioned anywhere, I just want to paste below the minutes from last week's .NET contributors meetings regarding this specific topic. Note that I'm just quoting, I don't personally share the same opinion on all of the points.

Editor unification (.NET integration as a plugin/extension) (switch to integration as gdextension or just always bundle it)

• Good way to move forward. Decouple C# support from the core. If anything is missing in GDExtension, it would be missing for other languages too.
• Would need to drop the entire interop code and rewrite it from scratch. Module uses a number of APIs which might not be exposed to GDExtension yet.
• C++ extension would not needed to be reloaded, and it would be able to reload C# assemblies just like the module does currently.
• Performance difference need to be assessed - attendees not convinced this would make a difference to the current approach.

It may even be that this unified editor idea is not in any specific roadmap, but just a common desire?

[akien-mga]

The unified editor with .NET support via GDExtension is being worked on (see https://github.com/raulsntos/godot-dotnet), but this is orthogonal to this proposal and thus off-topic.

The milestone is "when it's ready". When it's usable, there will be an experimental build with this, but we'll keep the separate "mono" build for some time to give the new approach time to mature, and users time to migrate their projects.

[GeorgeS2019]

https://github.com/godotengine/godot/pull/90510#issuecomment-2159522891

Still on the TODO list:

- Fix CI builds

[WIP 18 Aug 2024] There is now working example

https://github.com/migeran/libgodot_project/issues/1#issuecomment-2294996680

• Windows build
• Create a .NET/C# example using the new GDExtension-based .NET implementation.