StrataSource
Enhancement: Expose ImGui's API to AngelScript
What would this enhancement be for?
Tools
Describe your enhancement suggestion in more detail
ImGui in Strata has allowed replacing old VGUI debugs with much better and more flexible versions. While this is an overall improvement to the end user, the current ImGui implementation has a few issues here and there, or preference changes that I expect others and I to want to be able to have more control over.
Unfortunately, besides the styling for ImGui, most window customization and implementations are programmed into the engine which is closed source. When various issues with the ImGui occur, or tweaks are wanted to be made, GitHub issues requests have to be made, and I expect some issues to not be high priority based on that it could be a change that doesn't overall improve the engine, just allow for better debugging. One example would be a issue with the ImGui implementation of VProf where having the window too big makes ImGui exceed the VERTEX_BUFFER limit that's defined internally causing a engine crash rather than graceful handling if that is supported.
Rather than relying on internal code to be updated and waiting for a new engine build to be pushed, exposing the ImGui API to AngelScript will allow for people to make and edit ImGui for their own purposes, whether that be new tools or making edits to existing ones.
Caviot to this is that the exposed ImGui is only as powerful as it is in the engine code based on how much engine functionality is exposed to AngelScript. Currently CBaseEntity, various rendering debug, and TraceLine has been exposed, which is enough to work with to do most things if working with those with ImGui, however I expect more will need to be exposed overtime to do various other things. This does bring up the question with however should, or can be exposed for AngelScript for standard usage case or for ImGui.
Another caveat is how much of the "more specific" API of ImGui can be exposed. An example is that I would prefer to be able to have docking and viewporting features for the ImGui, so I can pull out various windows so they aren't on top of the game. However, that would require internally the DXVK implementation to support it, which I believe it currently doesn't because I am assuming you are all using the main branch of ImGui for simplicity's sake. But my point is that if it could simply go io.ConfigFlags |= ImGuiConfigFlags_ViewportsEnable and viewports just work.
There is also the other case of external ImGui libraries that are used along with ImGui, like ImPlot, ImGizmo, and other ImGui-enhancing libraries, which can be implemented internally, but externally isn't possible, I believe.
TLDR: ImGui's API should be exposed to AngelScript and/or possibly the other scripting systems like VScript and Panorama to give the end user more flexability with making debugging tools or editing existing ones and other various ImGui features.
I was just thinking about this too!
One example would be a issue with the ImGui implementation of VProf where having the window too big makes ImGui exceed the VERTEX_BUFFER limit that's defined internally causing a engine crash rather than graceful handling if that is supported.
This is an underlying limitation with ShaderAPI more than anything else. I don't think chunking ImGui vertex buffers into smaller sizes fixes it either, since the dynamic mesh will fill up regardless. More work is required to alleviate this issue.
Rather than relying on internal code to be updated and waiting for a new engine build to be pushed, exposing the ImGui API to AngelScript will allow for people to make and edit ImGui for their own purposes, whether that be new tools or making edits to existing ones.
Editing existing tools is not going to be possible, nor would it be useful in most cases. The primary reason why we create ImGui-based UIs in the first place is to provide a view on internal data structures, which we're not going to expose to scripting. Some of the ImGui windows are little more than wrappers around ConVars, and could be converted to AngelScript trivially if bindings were provided (These come to mind: CSM UI, Fog UI, the debug panel thing). More complicated UIs will need to stay in C++ (ex: Clustered light inspector, VProf UI, etc.)
There is also the other case of external ImGui libraries that are used along with ImGui, like ImPlot, ImGizmo, and other ImGui-enhancing libraries, which can be implemented internally, but externally isn't possible, I believe.
In the long term, I think supporting additional ImGui libraries would be good. I've used ImPlot before for some audio debugging stuff, but it's plagued by the same vertex buffer overflow issue as the VProf UI.
TLDR: ImGui's API should be exposed to AngelScript and/or possibly the other scripting systems like VScript and Panorama to give the end user more flexability with making debugging tools or editing existing ones and other various ImGui features.
These bindings would only be available in AngelScript. I don't see the value in adding these bindings to Panorama and we don't have the bandwidth to duplicate a bunch of work for VScript. Significant features like this should be scoped to AngelScript only, in my opinion.
While I appreciate the issue, the scope is far too large and ambitious. What I'd like to do in the short term is:
- Allow custom UIs in AngelScript, probably using an attribute like
[DevUI("mywindow")]to declare/register new ones - Provide bindings for a subset of the ImGui API
- TextEntry, Label, Button, tabs, tables, and other widgets
- Some safeguards will be required for these methods, because certain mistakes like using empty strings w/o PushID first will cause crashes.
- TextEntry, Label, Button, tabs, tables, and other widgets
- Convert some of the simple ImGui windows to AngelScript and include them in base p2ce
When I get home, I might split this issue into several
This is an underlying limitation with ShaderAPI more than anything else. I don't think chunking ImGui vertex buffers into smaller sizes fixes it either, since the dynamic mesh will fill up regardless. More work is required to alleviate this issue.
Couldn't a relatively simple fix be taking advantage of Child Window culling? Having a fixed size so it only shows a set of graphs. IMO that is what I would do if the window culling with graphs works that way (IIRC there were a few things that don't get culled by windows for reasons). I don't need to see all them at once pretty much. it depends on the users case and how they would prefer it I guess?
Editing existing tools is not going to be possible, nor would it be useful in most cases. The primary reason why we create ImGui-based UIs in the first place is to provide a view on internal data structures, which we're not going to expose to scripting. Some of the ImGui windows are little more than wrappers around ConVars, and could be converted to AngelScript trivially if bindings were provided (These come to mind: CSM UI, Fog UI, the debug panel thing). More complicated UIs will need to stay in C++ (ex: Clustered light inspector, VProf UI, etc.)
I understood going into this that there would be some things that can't be exposed internally. I understand that some stuff needs to be done internally for the sake of accessibility and implementation or something just simply can't be exposed. Still unsure what rules you have to follow with engine internals. I was just thinking some people want their UI a certain way, rather than how it currently is or want to improve it or add on to it. Maybe possibly contributing back those changes to the community through PR or what not.
In the long term, I think supporting additional ImGui libraries would be good. I've used ImPlot before for some audio debugging stuff, but it's plagued by the same vertex buffer overflow issue as the VProf UI.
I do remember some comment in the code or something in the ImGui docs. The solution to resolve that issue was to increase the defined macro limit or something like that? Horrible solution honestly, more just there in case you need to increase it for some debugging case. But at that point its more a issue of how it is being dealt with which should instead be fixed rather than just increasing a limit that could potentially be increased again when needed.
I am still confused on how AngelScript will be handled or how it works, Strata is the only game/engine I have seen it used or mentioned, so my assumption was like like Lua or VScript it's just a scripting language in C++ style that isn't compiled in. But if libraries could be a thing, I am unsure if its something that's gonna be internally linked or actually adding external libraries is dealt with and AngelScript is compiled or something like that.
These bindings would only be available in AngelScript. I don't see the value in adding these bindings to Panorama and we don't have the bandwidth to duplicate a bunch of work for VScript. Significant features like this should be scoped to AngelScript only, in my opinion.
Sorry, I should have worded that better. What I was implying with Panorama and VScript is just like how Panorama and VScript interact with one another, where it's simply just an event listener or calling a window to appear with a function. Minimal and simple, not exposing the entire API to each system, no need, too big, repetitive code. If someone wants ImGui, you do it through AngelScript, and you can have it be open by ConCommand, ConVar, or some VScript or Panorama function.
While I appreciate the issue, the scope is far too large and ambitious. What I'd like to do in the short term is: ...
Yeah, ok, re-thinking this over, doing every API function will be, in fact, a pain in the ass and just too much. Honestly having some abstraction to it would be beneficial to make things possibly even simpler, plus handling IDs internally rather than externally (which you aren't using io.ConfigDebugHighlightIdConflicts or the other error recovery flags to catch?) would be less problematic for the end user. But then again, it's pretty much abstraction, on top of abstraction, which just leads you to probably building some ID system to deal with an existing ID system. Although you could probably have something simple just to make sure they are somewhat unique, I don't know, up to you.
While it is just an abstraction over abstraction, it is still a good idea for each element to provide probably a less C-style way of dealing with ImGui and other potential improvments.
What ImGui windows can be converted to AngelScript is the idea I am trying to go for, so there is some control over what already exists, but again, not everything can be exposed like that.
Couldn't a relatively simple fix be taking advantage of Child Window culling? Having a fixed size so it only shows a set of graphs. IMO that is what I would do if the window culling with graphs works that way (IIRC there were a few things that don't get culled by windows for reasons). I don't need to see all them at once pretty much. it depends on the users case and how they would prefer it I guess?
Even with this the dynamic VB/IB will fill up. That VERTEX_BUFFER_SIZE value is the max number of verts that can be rendered in a single frame using the dynamic VB.
I'm fairly certain ImPlot has some type of culling enabled by default. I added a whole bunch of plots to a single window and didn't overflow the VB, even without child windows (most of the plots were scrolled off screen though).
Still unsure what rules you have to follow with engine internals.
We don't have to be too strict, just have to avoid implementation detail and whatnot. Binding things is quite tedious and it prevents us from refactoring/changing things later on, unless we wanted to break the script API.
I do remember some comment in the code or something in the ImGui docs. The solution to resolve that issue was to increase the defined macro limit or something like that? Horrible solution honestly, more just there in case you need to increase it for some debugging case. But at that point its more a issue of how it is being dealt with which should instead be fixed rather than just increasing a limit that could potentially be increased again when needed.
Yeah unfortunately even chunking the ImGui VB/IBs would not work. What we need is a separate mesh for ImGui rendering that is resized as needed. However, I encountered some issues with ShaderAPI the last time I tried to implement this sort of thing. This has definitely been on my mind for a while though; I just deemed ImGui functional enough to put this task in the backlog. (truthfully, I haven't hit a VB overflow with ImGui is quite a while)
I am still confused on how AngelScript will be handled or how it works, Strata is the only game/engine I have seen it used or mentioned, so my assumption was like like Lua or VScript it's just a scripting language in C++ style that isn't compiled in. But if libraries could be a thing, I am unsure if its something that's gonna be internally linked or actually adding external libraries is dealt with and AngelScript is compiled or something like that.
AngelScript is similar to Lua and Squirrel in that it's an embeddable scripting language. The way we'd add libraries is by compiling them into the engine, then writing AngelScript bindings for their APIs.
Do you have some ideas for custom ImGui windows that you'd like to implement? I'm thinking about this mostly in the context of debugging things in addons.