sammyfreg
poor performance attached to unreal editor by default
By default, Unreal Editor runs with this option enabled
This option, when the editor doesn't have focus, causes the editor to nearly stop updating. This will occur any time your NetImgui server app has focus. As a consequence, the frame events that UnrealNetImgui gets drawn to drop to near nothing, making interaction with the UI within the server app unusable.
Some possible fixes
Conditionally disable this option when NetImgui::IsConnected, like so
UEditorPerformanceSettings* Settings = GetMutableDefault<UEditorPerformanceSettings>();
Settings->bThrottleCPUWhenNotForeground = false;
The downside of this option is that it would basically keep this option disabled constantly, as just having the server running is almost always going to be connected. Also turning of this option keeps the editor eating a lot of CPU, as it attempts to run normally, causing your GPU/cooling to kick on as it normally does under load.
Another option potentially, rather than draw via FCoreDelegates::OnEndFrame, consider use a standalone timer callback, to fully decouple the 'framerate' of the editor, with the updaterate to netimgui, so NetImgui doesn't entirely circumvent the savings that are part of the bThrottleCPUWhenNotForeground option, and doesn't need to hijack the setting.
Perhaps the more notable bug in this situation is that the interaction with the widgets in the server window(clicks, etc) is so unreliable under this throttled situation?
Thank you. I have been meaning to explore accessing the CPUThrottle option in my next release, but the idea of not relying on FrameEnd might have some value in editor
I think there is something else going on, even with a timer based tick for editor, the click responsiveness in the server app is pretty bad.
Is there something inherent to this system that breaks down when decoupled from the framerate?
Seems like an issue with the server app itself. Is it capturing/caching the mouse click events in between updates so that it doesn't miss them? I have to long click to reliably trigger stuff. Quick clicking doesn't do anything much of the time.
I have confirmed that the timer approach calls Update at the expected rate. Tried up to 60 hz, but still the server interactivity is poor. Also confirmed that the OnEndFrame, drops to 3 fps when the editor doesn't have focus
OnEndFrame
Timer
I don't get why the interactivity is still flaky on a 60hz timer tick.
It should not be the NetImguiServer side, since it process content up to the framerate specified in its config (30fps by default) and this only arise when CPUThrottle is on in Unreal Editor. Maybe the communication thread gets throttled too somehow?
See if EndFrame() in 'NetImgui_Api.cpp' gets called at 30/60fps (should be with your timer callback change) and then check in CommunicationsHost() in 'NetImgui_Client.cpp' which runs in a separate thread to exchange with the Server.
Looks like EndFrame starvation is still occuring
In/Out is the Communications_Incoming and Communications_Outgoing
The top sections are the EndFrame counts when the app has focus, the ones below are when it doesn't have focus
Looks like the issue is related to NETIMGUI_FRAMESKIP_ENABLED, passing 0 to NetImgui::NewFrame seems to fix it
This option is tied to mbValidDrawFrame and this is set by ProcessInputData. Does that mean if there is no input captured in the frame, it's not a valid draw frame and so it won't send with that option?
I could see how the app would not have input state if it doesn't have focus, but I don't understand why that would prevent the draw frame.
This InputData is received from the NetImguiServer, not Unreal. When there's no pending input, it skip drawing content in the editor/game to preserve CPU since it will be discarded. With frameskip disabled, it makes sures there's always a valid imgui context to draw into, but the content is discarded, never sent to the server.
Check that Communications_Incoming_Input() is called every ~33ms (or whatever fps you set on the server). This is called on the communication thread, when receiving data from the server. The server never change its speed, after receiving a draw command from the game, it waits until 33ms has passed to send a new input back to the game, triggering a new draw. If we are not receiving it at that rate, the only thing left I can think of at the moment, is that the Epic TCP/IP com library also gets slowed down when CPU throttle is active?
Without having a new Input received every 33ms, then Communications_Outgoing_Frame should also not be having a pending drawcommand send back to the server either at 33ms.
Incoming_Input rate is steady
The hasNewInput in ProcessInputData is what drops
I'm not sure why the Incoming_Input actually doubles in rate when the editor doesn't have focus while the HasNewInput drops to 3-4, but I guess that shows the network thread isn't being starved on the unreal side.
Based on my understanding, this means the input messages double in rate when the server window has focus, but it makes no sense as to why hasNewInput would drop, bc it looks like those are tied together. Incoming_Input sets client.mPendingInputIn every call, and that's the basis for hasNewInput, so as far as I can tell, those numbers should match.
Only thing possible it looks like to me would be if a bunch of input events are coming in and just stomping each other before one can be processed. Input isn't processed as a queue, so if multiple come in they would just stomp each other.
Confirmed, lots of input stomping
Didn't you say there should be some sort of 2 way lock stepping thing or did I misunderstand that?
I'm currently setuping myself to test this, but the question is why ProcessInputData is still called at a slower rate if it doesn't have time to process input data.
The ProcessInputData call rate stays steady at 60
Only thing I can figure is that the network buffer is filling up with a bunch of inputs, and since data is pulled out of the socket until there is nothing left in Communications_Incoming, we get a ton of input message stompage in between frames of ProcessInputData. Any reason not to push these into a queue instead?
Simplicity :)
The received input are bitmask letting us know if a key is up or down. I'm not sure what would happen to suddenly insert 10 frames of key input events into 1. But it might be worth it after all.
That said, it is not normal that ProcessInputData() and Communications_Incoming_Input() are both called at a high rate but the processed input isn't. Your number of Incoming_Input is with valid data received and assigned?
Well, ProcessInputData is gated by FNetImguiModule::Update calling NewFrame, called by the 60 hz timer in this case(even slower without the timer when not focused), while the comm thread runs freely. That inherently creates a race condition unless there is some sort of network level message exchange that keeps the systems in some sort of lockstep.
Yes, it seems like I will have to improve the input reception to buffer them until processed.
I'm currently looking at the slow frame rate.
It seems to me a queue is the right approach, in principal. Otherwise any sort of stomping or input dropping will cause unreliable input in the server app. However, the big worry with queuing is what if the app can't keep up with the input rate?
Clearly we're getting more input right now through the comms than it can process in the main loop, so it's possible the EnQueue count will outpace the Dequeue count, unless you while(Dequeue) each frame in the client side to clear the buffer each frame. That sounds proper to me, as you need to process each one through the full set of ImGui draw pipeline to pick up on all the UI state changes each individual input causes, but you don't want to send a render frame back to the server for all of them, just the last one
I believe I found the issue for the low responsiveness with the Timer method. When unfocused, the callback timing is not steady at all. Overall seems to match the requested speed, but when inspecting the Elapsed time between call, I see about 10 quick call, then a long pause (timing in ms):
| [0] | 0.000600000028 | float
| [1] | 0.000500000024 | float
| [2] | 0.000600000028 | float
| [3] | 332.980194 | float
| [4] | 0.188800007 | float
| [5] | 0.000799999980 | float
| [6] | 0.000300000014 | float
| [7] | 0.000199999995 | float
| [8] | 9.99999975e-05 | float
| [9] | 0.000199999995 | float
| [10] | 0.000199999995 | float
| [11] | 0.000199999995 | float
| [12] | 0.000199999995 | float
| [13] | 333.130402 | float
| [14] | 0.564000010 | float
| [15] | 0.00200000009 | float
So, it appears something else should be used if available, or just disabling the CPU optim whiled connected. There's still the problem of better input enqueuing, but it is not the main issue observed here.
Note: Looking at the Engine loop a little closer, I do not believe there's a way around disabling the CPUThrottle and keeping things on the GameThread.
I suppose that's coming from UEditorEngine::ShouldThrottleCPUUsage
I had initially only seen the first reference to bThrottleCPUWhenNotForeground which pauses the viewport
Looks like they have a hook to get around that, see ShouldDisableCPUThrottlingDelegates
This can check if Netimgui is connected.
This will allow the CPU throttle to bypass without missing out on the viewport render throttle.
Yes, since the timer won't work. I was pulling my hair trying to understand how I could have 30fps in the Update, 60fps in the reception of input and still have a bunch of null pending input :)
This simple code addition (executed after the engine has been properly init) does the trick. I will include it in the next release. Thank you for the pointers.
#if WITH_EDITOR
GEditor->ShouldDisableCPUThrottlingDelegates.Add(
UEditorEngine::FShouldDisableCPUThrottling::CreateLambda([](){ return NetImgui::IsConnected();})
);
#endif
Where are you calling that from? In my project the GEditor isn't initialized yet(module load order) if called from FNetImguiModule::StartupModule
For the sake of avoiding these issues, you might want to set that GEditor delegate inside of a FCoreDelegates::OnPostEngineInit
#if WITH_EDITOR
FCoreDelegates::OnPostEngineInit.AddLambda([]
{
GEditor->ShouldDisableCPUThrottlingDelegates.Add(
UEditorEngine::FShouldDisableCPUThrottling::CreateLambda([](){ return NetImgui::IsConnected();}));
});
#endif
Yes, I moved some plugin init that included this new throttle callback, to a new method attached to the PostEngineInit delegate.