godotengine
Expose a zero allocation API to Godot C# bindings
Describe the project you are working on
Godot C# bindings
Describe the problem or limitation you are having in your project
For the past weeks, I've been discussing with several Unity users intending to move to Godot C# regarding dealing with the C# garbage collector.
The most common complaint I hear from users is that, in Unity, allocations can trigger unexpected GC spikes into the game.
In Godot, we target to make all of the high performance APIs (those that intended to be called every frame) not allocate any memory, so theoretically the GC should not be a problem. Additionally, Godot starting from 4.0, uses the Microsoft CoreCLR version of .net, which also supposedly has a better garbage collector than Unity.
But in all, after several discussions with Unity users, neither is enough reassurance for them, and they would really feel safer if Godot exposed a zero allocation API.
Describe the feature / enhancement and how it helps to overcome the problem or limitation
The idea of this proposal is that Godot exposes zero allocation versions of many functions in the C# API, that users can use if they desire.
Technically, this could be done from the binding generator itself, without breaking compatibility, and without doing any modification to Godot itself.
Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams
WARNING I am not familiar with C#, so take this as pseudocode.
Imagine you have two functions exposed as to C#:
void MyClass.SetArray( Vector2[] array);
Vector2[] MyClass.GetArray();
This works and is pretty and intuitive. However, it has two problems:
- GC is allocated on return
- Memory is copied to Godot native formats every time there is a call.
The idea is to add NoAlloc versions, which can be generated directly by the binder automatically when required:
void MyClass.SetArrayNoAlloc( Godot.Collections.PackedVector2Array array);
void MyClass.GetArrayNoAlloc(ref Godot.Collections.PackedVector2Array ret_vec2_array);
This way, we solve two problems:
- Using the Godot native types, no conversion happens between C# and C++, the Godot array can be used directly from C#, and even reused to pass multiple times as argument.
- The return version takes an existing Godot array and fills it.
As a result, the Godot API can be used entirely from C# in a way so, if this area is performance critical somehow, we ensure no memory copies, safe native access to Godot internal data and ability to control what goes to the GC.
The idea is to provide these specialized call syntaxes for functions using packed arrays, arrays and dictionaries.
If this enhancement will not be used often, can it be worked around with a few lines of script?
N/A
Is there a reason why this should be core and not an add-on in the asset library?
N/A
I like this part:
"Technically, this could be done from the binding generator itself, without breaking compatibility, and without doing any modification to Godot itself."
This proposal would give users more control over their game's performance while the tradeoff on Godot's part is essentially zero. (Sure, some dev time to get it set up.)
It sounds like a lot of pros for hardly any cons. Am I reading it correctly like that?
Would this automatic re-binding work for APIs that have variant Arrays in them, like ArrayMesh's add_surface_from_arrays? Or would add_surface_from_arrays continue to take a Godot Array with C# arrays inside it with no alternative? (To note, some of the entries in the Array passed to add_surface_from_arrays are of determined-at-runtime types - the custom-format arrays like ARRAY_CUSTOM0 etc).
@wareya I think the main problem here is more along the lines of the raycast function rather than a GC one. If you want to build and update geometry every frame, Godot has API in C++ for this (as example the one used by softbodies), but its not exposed to the binder due to limitations of it. Solving that issue should solve this one too.
Are there any actual confirmed cases of GC stalls caused by this? Don’t get me wrong, but getting a huge array of vectors from the engine is not a common task, it goes to Gen1 type GC queue and shouldn’t affect performance anyway. Submitting arrays to the engine is an another story but still can be improved easily with pointers - pass a pointer to the array instead of doing stackalloc and copying the contents in manage score but copy it in CPP part instead. My concerns is that Godot.Collections are not so common and flexible as built in CLR types and with it we are moving farther from vanilla .Net to a custom language dialect. P.S. Surely, if you can implement a NoAlloc version in a few clicks, we can give it a try and I might be wrong and it will be even handlier.
@reduz I don't rebuild geometry every frame all the time when standing still with the world fully loaded, but I do while the world is loading and also when the player travels into new chunks. I do it on a separate thread, streaming everything in as fast as possible, so while it doesn't cause a lower framerate or any stuttering, a performance improvement in removing the extra copy would be noticeable to players as being able to turn up the render distance higher and distant lands becoming visible more quickly. Despite being an unusual edge case, it makes the extra copy when converting the C# arrays into Godot packed arrays worth worrying about to me for a reason other than framerate or loading time impact.
@Nomad1 Godot internally does not use C# arrays, so you have to pick. You use the pretty dialect and stand copies and GC issues, or you use the Godot dialect and ensure maximum performance.
I am pretty sure that not all your code will need to use the Godot dialect, so it should be reserved for cases where you absolutely want the performance.
Nice! I'm following this discussion and this proposal has the potential to touch some hearts.
I think it's out of the scope of this discussion, but would be nice to see the boundaries of this zero allocation API. Having a list of the mapped functions. So the work to be done would be clear.
And I think it's not the moment to think about the details of the implementation, but what do you think of using the same structure of the current API with a different namespace? Something like
void NoAlloc.MyClass.SetArray( Godot.Collections.PackedVector2Array array);
void NoAlloc.MyClass.GetArray(ref Godot.Collections.PackedVector2Array ret_vec2_array);
Great stuff, very much in favour, and I really appreciate that you heard the feedback and opened the proposal. Proposal coming from Juan will have a different reach than one made by a random "Unity-refugee".
Obvious next step would be a C# profiler in Editor, but that's a separate thing and I'm sure will be tackled in the future (for now the JetBrains one has to be enough).
@Nomad1 Godot internally does not use C# arrays, so you have to pick. You use the pretty dialect and stand copies and GC issues, or you use the Godot dialect and ensure maximum performance.
C# Vector2[] in memory is stored exactly the same as C++ Vector2*. We can pass it to the native code for free, with zero copying or using Godot.Collections.Array<Vector2>. There won't be any issues unless you take the pointer and hold it longer than needed. And even if you intend to make a hard copy of that data in C++ code, it will be out of GC scope. So I'd say it's generally better than using your own collection types and losing things like Span<>, BlockCopy, natively optimized iteration, etc.
@Nomad1
C# Vector2[] in memory is stored exactly the same as C++ Vector2*. We can pass it to the native code for free
Except for some specific cases where Godot does take pointers (in very high performance APIs such as RenderingDevice::buffer_update as example), in far most cases it uses reference counted arrays because its far easier and safer to pass around information this way, so even if Unity can pass a raw pointer, this is not useful to Godot.
Hence, the way to do these things efficiently in C# in Godot is to use the Godot collections.
Is the array always big enough or do you need to provide a hook in the api to communicate its too small, i.e.
bool MyClass.GetArrayNoAlloc(ref Godot.Collections.PackedVector2Array ret_vec2_array, ref int size);
// or if PackedVector2Array knows its size:
int MyClass.GetArrayNoAlloc(ref Godot.Collections.PackedVector2Array ret_vec2_array);
This is very cool! I'm glad to see this being embraced!
As the person who started all of this mess for you (sorry) I'd be happy to work on the design and maybe implementation side of this if it would help. I've got quite a few ideas about how to do this style of API nicely.
The basic idea here is very similar to what I had imagined, which is lovely - particularly where it's handled automatically in the binding.
I'll put some more detailed proposals down sometime soon if they're welcome.
@reduz
C# Vector2[] in memory is stored exactly the same as C++ Vector2*. We can pass it to the native code for free
Except for some specific cases where Godot does take pointers (in very high performance APIs such as
RenderingDevice::buffer_updateas example), in far most cases it uses reference counted arrays because its far easier and safer to pass around information this way, so even if Unity can pass a raw pointer, this is not useful to Godot.Hence, the way to do these things efficiently in C# in Godot is to use the Godot collections.
Well, my approach was never about adopting the raw pointers internally, but about adding C++ overloads that can make a Godot.Collections.Array from data pointer instead of numerous calls to monsters like godotsharp_packed_vector2_array_new_mem_copy. That's how we can achieve performant interop without extra pressure on GC or filling the code with collections created in C++ and passed back and forth between unmanaged and managed parts. However, I understand that it will be nowhere near your solution by the amount of changes required.
I agree with @Nomad1 that this is a good opportunity to get some additional performance wins (in both C# and GDE) by allowing some internal allocations to be optionally skipped in favour of external preallocated buffers.
For these things, it actually should be possible to use the C# collection types.
Well, my approach was never about adopting the raw pointers internally, but about adding C++ overloads that can make a Godot.Collections.Array from data pointer instead of [...]
Godot's array types are dynamic arrays, not static arrays. They need to be able to resize themselves, which means that the pointers inside them have to come from the same allocator that they themselves use. This is a UB minefield and probably wouldn't even be safe for C# code.
(They would need to come from the same allocator even if they were static arrays, since they have to be able to destruct themselves when their last reference expires, but the possible workarounds for that are less dangerous than with full reallocation.)
Well, my approach was never about adopting the raw pointers internally, but about adding C++ overloads that can make a Godot.Collections.Array from data pointer instead of [...]
Godot's array types are dynamic arrays, not static arrays. They need to be able to resize themselves, which means that the pointers inside them have to come from the same allocator that they themselves use. This is a UB minefield and probably wouldn't even be safe for C# code.
Those are created and resized in C++ code with a consistent allocator: https://github.com/godotengine/godot/blob/master/modules/mono/glue/GodotSharp/GodotSharp/Core/Array.cs So it's safe and nice, but not super-fast since every Add or Count call is an Interop. I'd love to avoid those classes in ultra performance-critical parts, i.e. when you are animating the water on the CPU and sending a bunch of vertices every frame. However as I said before, I don't think that it would make a big difference anyway. Will need to do some profiling first.
So it's safe and nice, but not super-fast since every Add or Count call is an Interop. I'd love to avoid those classes in ultra performance-critical parts, i.e. when you are animating the water on the CPU and sending a bunch of vertices every frame. However as I said before, I don't think that it would make a big difference anyway. Will need to do some profiling first.
I feel like something similar to this could be achieved by reading the raw pointer out of the array and making sure not to destroy or modify it array until you're done using the raw pointer, rather than by trying to construct a godot array out of a raw pointer.
So it's safe and nice, but not super-fast since every Add or Count call is an Interop. I'd love to avoid those classes in ultra performance-critical parts, i.e. when you are animating the water on the CPU and sending a bunch of vertices every frame. However as I said before, I don't think that it would make a big difference anyway. Will need to do some profiling first.
I feel like something similar to this could be achieved by reading the raw pointer out of the array and making sure not to destroy or modify it array until you're done using the raw pointer, rather than by trying to construct a godot array out of a raw pointer.
I doubt it's possible. We can only guarantee that the raw pointer is immutable during the single call and within fixed(Vector2 * ptr = &array) { ... }. But that data should be copied to some other collection anyway. Right now it's done in the managed code in the GodotSharp assembly, I recommend doing it in the unmanaged code, Juan proposes to use unmanaged collections from the start. All three approaches have flaws but Juan's one is easily achievable.
@wareya @Nomad1 Consider that what I am describing in this proposal is not possible yet because the Godot collection types are not properly exposed in C#. They need to be rewritten to work as I describe using the new extension API in Godot.
The is because C# support in Godot is quite old (from the Godot 3 days) while Godot 4 uses a new extension API that is far more efficient and supersedes all the glue code that Ignacio had to write from scratch for mono.
Ideally, in the coming months (after getting current C# to work on mobile), C# support in Godot will most likely be rewritten to use the new Godot 4 extension API (this will make performance 100% optimal, allow rewriting any part of Godot in C#, even renderers, add classes as native classes, unify the .net editor with regular editor, etc) and this will be implemented there.
Consider use Span<T> as return type. That could be nice wrapper around native arrays
I don't know if NoAlloc needs to have any special designation in the name; would a native collection as an argument be enough to make that functionality implicit?
I don't know if
NoAllocneeds to have any special designation in the name; would a native collection as an argument be enough to make that functionality implicit?
Godot's API gets exposed to languages that don't have argument-list-based overloads, and you might want the API to be consistent between them and languages that do. I'm not sure if NoAlloc is the best way to do it, but it makes at least a little sense.
Godot's API gets exposed to languages that don't have argument-list-based overloads, and you might want the API to be consistent between them and languages that do. I'm not sure if NoAlloc is the best way to do it, but it makes at least a little sense.
Oh yeah, in the native binding format it makes sense to have a designation. I was thinking more in the context of C# specifically, and if it could suffice with a uniform name on the surface
Are there any actual confirmed cases of GC stalls caused by this? Don’t get me wrong, but getting a huge array of vectors from the engine is not a common task, it goes to Gen1 type GC queue and shouldn’t affect performance anyway. Submitting arrays to the engine is an another story but still can be improved easily with pointers - pass a pointer to the array instead of doing stackalloc and copying the contents in manage score but copy it in CPP part instead. My concerns is that Godot.Collections are not so common and flexible as built in CLR types and with it we are moving farther from vanilla .Net to a custom language dialect.
I do not believe that these GC stalls happen, or are a problem at all.
I believe that this proposal stems from the recent social media pressure. Since 2 weeks ago, there's been a very judgmental tone online from a few of the recent "Unity refugees", about Godot's supposed "poor performance". The demands they've been giving can roughly be summarised as:
- Godot should have the best performance possible, at any cost, even that of good design
- Godot must have the best performance in C#, no care for C++ at all
While this mentality would make sense in Unity, where the codebase was handled by Unity and was never the community's responsibility, and it only supported C# for coding, it doesn't at all in Godot, where the community has to maintain a large and complex codebase on a limited budget, and where the usability/performance ladder goes:
- GDscript, VM, instant iteration, quick to write and read, highly integrated, mild performance
- C#, compiled, slower iteration, access to a massive ecosystem of libs, great performance
- C++, compiled, slowest iteration, best performance
In the past days, attempts to explain these design aspects of Godot to these demanding Unity users has been unfruitful. The main point of failure in these attempts to explain is sadly that, well, they don't care. While some Unity former users have made amazing work with Godot's features, and one absolute monster has transferred the entirety of his game's logic in C# from Unity to Godot in 14 hours, for some Unity devs, it's been a full public judgement of Godot's codebase before talking to anyone from Godot, or even asking why are things made the way they are.
Articles have been written and have taken an extremely small part of the engine and made extensive judgements with hyperbolic words such as "terrified, limited potential, dramatically and systematically worse performance". These articles have the same validity as me going to Unreal and picking the worst, most unoptimised part of their stack and claiming that its "incredibly unoptimised"...yet a very large number of people have taken these articles as a legitimate study of Godot's performance in depth and ran with it.
Besides the quick, and mostly false, judgements, the bigger problem is that the claimed pain point of these people, performance, is, at the risk of triggering them, of secondary importance for most of gamedev, and only one of the main goals of Godot's design. As I said above, there are 3 languages that Godot lets you develop with, and each responds to a different kind of need in gamedev:
- Highly tweakable components (player characters, gameplay feel, special effects, enemies, AI, immediate environment...)
- Regular, strong performance components (systems, game background logic...)
- Mandatory, best performance components (render/physics, very large amount of entities, high perf systems...)
This division of labor is because gamedev isn't about slogging through code optimisation like you're making YAPSQL but about a slew of different trades working together (artists, animators, game designers, programmers, render/physics specialists, tool engineers, etc). GDscript is meant to help artists, animators, designers, iterate as fast as possible, with a simpler, readable syntax, running on a VM, with tight integration, hot reloading and extremely practical iteration. C# is to open up generic purpose, high performance code to a very large ecosystem of libraries and tools, for programmers to add any logic that their game could need. C++ is to get the best performance possible, in the most performance critical areas of the engine, for render, physics, or top performance code. And GDExtension, even without modifying the C++ source code, allows to extend into almost any language, so you can even get your own flavor of code.
The very design of the engine is centered, all the way down to its multiple languages, to cater to all these needs as best as possible. Each layer fit for purpose, with performance and usability being yin and yang. This in my opinion, is Godot's greatest strength. It's what sets it apart from any other engine. Godot intelligently balances out performance, usability, productivity, amount of targets, and manages to remain greatly sustainable and multifunction.
However, and without mentioning any one individual in particular, this "star feature" of Godot was completely ignored in favor of a highly ideological, inapplicable vision of programming and...well, in favor of Unity. The given justification of "performance", which is waved around like a flag, falls apart quickly when their complaints are about non-performance critical parts of Godot, or when you ask them to directly use the engine's C++. The former is rejected on ideological grounds: "everything should be high performance". Even if it needlessly complexifies and damages the engine's code clarity. The latter on convenience grounds: "I don't know C++". Asking for the greatest C# performance and NoGC policies when Godot's C++ and GDExtension are right in their noses is, to say the least, out of touch.
In fact, they are so out of touch that we're now to the point that rather than them dealing with C++'s manual memory management, we're now being pressured into considering a proposal where we have to build a whole new copy of the C# bindings to walk around C#'s GC, instead of telling them to use the base engine language that already has none. Because C# was their main horse in Unity, and so they demand that it remains it now, and Godot is supposed to reimplement a custom version of something that already exists.
Whether you look at this situation from a usability, productivity, code clarity, or performance standpoint, I do not believe that this proposal responds to any needs of Godot, but merely satisfies the demands from a group of people who intend for Godot to patchwork itself into replacing Unity for them.
I do not believe that this extra layer of complexity to C# is in the interest of Godot, or fitting to Godot's brilliant design. If truly, a C# layer with strong performance isn't good enough, then C++ is right open to a user. Or GDExtension. Instead of flatly stating why Godot is what it is, we're conceding to social media pressure from a vocal minority, and considering denting Godot's excellent design, for people who did not and do not care to understand Godot, did not care to try it for the vast majority of them, and who want:
- Unity like constructs
- Unity like design
- Unity like habits
- Unity like language
Even if it brings:
- needless complexity in C#
- needless performance optimisations that C++ already offers
- unwanted divergence from Godot's clever division of labor
- a precedent that C# should become the dominant language of Godot
This isn't tribalism, nor is it personal. It is a raison d'être. Godot is designed in the best way that I can think of. Unity was not. This proposal is encouraging that we start abandoning Godot's greatest strength to appeal to Unity users who did not even care enough to try Godot, let alone adapt to it, and instead, come with quick judgements and a demanding stance. The numerous other users that have engaged positively haven't been close to this demanding, and instead lay praise on Godot regularly, whether for its usability or its performance or design.
Godot has always been open to bettering itself. I don't believe that this is about bettering it. This is departing from its clean design to satisfy people who do not even try to understand Godot and judge it "wrong" for not being Unity.
I believe that it is a very bad idea to support this. What this will effectively do is set a precedent where, instead of imposing Godot's design, we accept to harm the design in the name of appealing to Unity users who will certainly keep pushing further for Godot to become even more like Unity. Complaint after complaint, we will have to argue Godot's design against Unity's design, which will only fracture the community.
While there is certainly some good to take from Unity, dismissing the core design of Godot is not something we should encourage or accept. It'll lead to the original Godot users pushing for Godot, while the new ones will push for Unity. When the Unity users who started out by writing judgmental articles without knowing Godot well are encouraged with "Unitifying" it, we're marching against Godot's long term interest.
I believe that we should accept that in the implosion of Unity, we have gathered a massive amount of new users, support, and love, and that some people will not even care to look at Godot for what it is, and should be politely told that it's probably not the engine for them. And if they feel like this is a justification to throw shade on it and make judgmental claims without having even tried it, well, they were doing that from the start. Caving in is not going to stop that attitude. If anything, after this request, there will be another, and another, and more and more, and that any rejection will be another round of public complaints and judgements, until we cede to them again.
Is this proposal made in the interest of Godot? Is it following the proper division of labor and design principles of Godot? I think not.
Is it meant to concede to this vocal minority of Unity devs who have not engaged with Godot, but merely demanded that it change to suit them better? Will this minority be encouraged to demand even more and keep reshaping Godot to fit the preferences they have for Unity? I think so.
Please consider this along with the technical aspects of this proposal.
@AwayB Ok, so let us assume for a minute that the presuppositions on the technical merits of the proposal that you are making here (that those merits are near to non-existent) are true (not that I agree with these presuppositions, but it should make things simpler).
...So what if this proposal is "meant to concede to this vocal minority of Unity devs who have not engaged with Godot". Why is it supposed to matter whether there is much a particularly large technical merit to the proposal ? From what I've read in this thread, the technical cost of the proposal is near-nil. Is that such a big cost to take in exchange for getting a massive amount of Unity devs over to Godot ? In other words, it seems to me like even the smallest technical merits for the proposal seem to me like they would be enough to accept it, given how little it costs to accept compared to how much good it can make for the Godot project, even if only to see if it is worthwhile in practice. Is the mere existence of a zero allocation C# API next to the other bindings going to make the rest of them worse ?
The only arguments I can see in your post that could counter this would be that either this will magically send Godot down some hyper slippery slope that will instantly destroy the engine, or that the engine being popular is actually a bad thing. Neither seem plausible to me. In particular, could you detail how this is supposed to be "dismissing the core design of Godot" ?
Also, one last thing. I'd like to point out one quote from what you've said:
I do not believe that this proposal responds to any needs of Godot, but merely satisfies the demands from a group of people who intend for Godot to patchwork itself into replacing Unity for them.
What does the "needs of Godot" refer to here ? Is Godot some kind of abstract entity completely separate from what its users want ? Because that sentence just sounds like "I don't think Godot should answer to the needs of its users" to me.
The only real counter-argument I could find was this:
we're now being pressured into considering a proposal where we have to build a whole new copy of the C# bindings to walk around C#'s GC, instead of telling them to use the base engine language that already has none
But they themselves admitted that C# allows for faster iteration than C++, and I think we can all agree that it is easier to use as well. Obviously developers using Godot should be allowed to pick C# over the other two languages, and whether they come from Unity or elsewhere is irrelevant to the amount of support they would get from the engine.
And while there is undeniably an influx of Unity developers, the alleged "social media pressure" from these developers is far outweighed by the doubling of monthly Godot funding that came as a result of all of this. I assume that is more than enough to develop and maintain the relatively small part of the codebase that would be required to implement this proposal.
Technically, this could be done from the binding generator itself, without breaking compatibility, and without doing any modification to Godot itself
Ideally, in the coming months (after getting current C# to work on mobile), C# support in Godot will most likely be rewritten to use the new Godot 4 extension API (this will make performance 100% optimal, allow rewriting any part of Godot in C#, even renderers, add classes as native classes, unify the .net editor with regular editor, etc) and this will be implemented there
Both quotes from reduz, the first being from the OP. This isn't a matter of overhauling a working system to appease a vocal minority, it's recognizing something that can be implemented in a rewrite that's going to happen anyway. The C# system as it currently exists is inherently legacy, there's nothing antagonistic in saying that
@AwayB
Is this proposal made in the interest of Godot? Is it following the proper division of labor and design principles of Godot? I think not.
Actually, I want to second this statement, despite several negative emotions that some people put underneath that post and the obvious prejudice against Unity newcomers (among whom I consider myself to be, since my previous engine was Unity, ignoring the fact that I used it only for rendering). GC stalls might be a problem or might be not. But I really doubt that it's that critical to the engine auditory as a whole and I'm sure that effort could be put elsewhere with much more benefit to the community. Of course, it would be nice if something could be done pretty much for free (= a few lines change in the binding generator), but if it takes more than a day, I'd vote to skip it. And yes, there are bigger challenges on the horizon, not to speak of well-known issues like a lack of mobile platforms.
Technically, this could be done from the binding generator itself, without breaking compatibility, and without doing any modification to Godot itself
Ideally, in the coming months (after getting current C# to work on mobile), C# support in Godot will most likely be rewritten to use the new Godot 4 extension API (this will make performance 100% optimal, allow rewriting any part of Godot in C#, even renderers, add classes as native classes, unify the .net editor with regular editor, etc) and this will be implemented there
Both quotes from reduz, the first being from the OP. This isn't a matter of overhauling a working system to appease a vocal minority, it's recognizing something that can be implemented in a rewrite that's going to happen anyway. The C# system as it currently exists is inherently legacy, there's nothing antagonistic in saying that
reduz stated that he is now under constant pressure from the newcomer community and social media. The fact of this proposal might be a sign that he is already thinking about diverting the energy to follow the vox populi instead of his own roadmap of engine development. I don't think that's a good sign. I was on this road once and it's paved with good intentions but leads to some other place.