SDL
SDL copied to clipboard
libsdl-org
Mainloop control inversion
I want to start this by saying I don't like this direction that various platforms are taking, but they are taking it regardless of my feelings, so we should talk about this for SDL3.
We have seen several platforms that would prefer a control inversion in the app model: instead of the app having a main loop that collects input, thinks, then renders, they want the apps to provide a callback that fires once per frame, and the platform handles the main loop.
The notable one in SDL2 is Emscripten, where you have literally no choice: your SDL-based app can share most of its code, but you have to have some ifdefs that provide a callback function and return from main() to start your actual app lifecycle. If you don't, you can make OpenGL calls and call SDL_GL_SwapBuffers, but nothing renders (and other problems) unless you use the callback.
Another up and coming platform is RetroArch, where it would be amazing if any SDL-based app could become a "libretro core" and treat the libretro API as the platform layer...but it also demands this callback interface to function.
Other platforms don't require this but prefer it (iOS or Android? Wayland? I can't remember) since it can reduce battery usage.
It might even solve #1059 if SDL worked this way.
I would not want to force this on SDL users, but it might be nice to find a way for apps to be able to adapt cleanly to this way of thinking and/or have the option to work this way.
I do not have a solution--there might not be an acceptable solution--but in the initial SDL3 window, this is the time to talk about this sort of thing.
The notable one in SDL2 is Emscripten, where you have literally no choice
My stuff that uses SDL already supports this model for that reason - it actually wasn't that complicated at all to change my code to make it work, but I don't know how typical that is.
On a practical level what sort of pros/cons does that approach have (aside from power usage as was already mentioned)? I suppose some control over framerate is given up?
I suppose some control over framerate is given up?
Absolutely, but I'm pretty sure all the ones doing this at the system level are doing it so you can only render at the monitor refresh rate for battery life, CPU/GPU workload, and cooperating with the compositor.
(I wouldn't be surprised if RetroArch clamps to vsync too, but it only has application-level control over this.)
More aggressive options would include what macOS does with App Nap...if you aren't visible, they have the option of not calling the callback at all.
The biggest con is that it requires apps to make at least mild modifications, although in most cases it comes down to a few #ifdefs around the startup code to either set up the callback and get out, or go into a main loop that breaks when it's time to terminate the application. In "well-designed" apps, it ends up looking like:
int main() {
// other setup stuff here
#if USE_CALLBACK_RENDERING
register_rendering_callback(render_one_frame);
#else
while (!time_to_stop) {
render_one_frame();
}
#endif
...and that's it.
BUT...even well defined apps can have That One Function Somewhere That Blocks:
// this gets called 100 stack frames down, passing through a scripting language on the way, etc.
void confirm_question() {
int answer = -1;
while (answer == -1) {
answer = render_are_you_sure_message();
}
do_something_with_answer(answer);
}
...and if you have a bunch of these sprinkled around your codebase, you can be in for a world of hurt trying to convert your program.
Which is why, even if I wasn't a control freak about things, would never recommend this being the primary way that one writes SDL programs. I'm just trying to decide if there's a nice way to make either approach work well.
Maybe it's as simple as "OPTIONALLY, you can call SDL_SetRenderCallback, but the usual ways usually work too" ... on platforms that prefer this, we take advantage of the system-level APIs. On platforms that don't offer this, it's trivial to implement it inside SDL itself.
On platforms that require it, you either need to use it for your app, or have those #ifdefs in place to use SetRenderCallback on those platforms, like you would have to do anyhow.
I dunno, I'm just thinking out loud here.
The only way I know of to "magically" turn a regular loop into a callback is to use some sort of threading. (generators, fibers, or OS threads) That's obviously fraught with peril in this case. Fibers aren't portable, and threads are generally off the table too for many reasons. I don't think there is any other way to escape turning the loop into a state machine without making the user do it in their own code.
Android is running the usual SDL way (no callback). because in fact we create a SDL thread beside the android activity.
But it could also be implemented with a callback ( https://developer.android.com/reference/android/view/Choreographer.FrameCallback ). not planning to use it though ...
The great force of SDL is to write once your app, and it can run without any change to most platform... so it if you need to two different way, it starts to be less convenient .
( I don't mind one or two #ifdef) but if you have nested windows / dialog box with it's own event polling , it's more difficult.
(100%, if we do something here at all, it won't be to change this to require a callback. That's a total non-starter.)
On iOS, you actually need to run your code in the display callback to correctly interoperate with other UI components. That's why we have SDL_iOSSetAnimationCallback()
Unfortunately I don't think there's a magical solution here. If we required a callback we can make all platforms behave identically. If we start the application code on its own thread, we can make all platforms behave identically (but introduces multi-threading issues and potential latency). But if we try to keep everything on the main thread and have the application drive the frame loop then we're stuck with what we have now.
I wonder if this is an opportunity to beef up the SDL_main functionality. Maybe we provide a header-only library that allows you a few ways to start your main function or set dispatch callbacks, and existing applications can call into SDL functions the way they always have?
Dumb thought: So fibers might not be portable enough to be used everywhere, but they are relatively easy on ARM or x64 platforms and could be an option provided by SDL_main. They are complicated on Emscripten/WASM though. Asyncify exists, but I don't know much about it. Sounds like it has a lot of compile time overhead.
I believe asyncify ends up running one's code as an interpreted bytecode (in addition to WASM itself being a bytecode format as well), so it can start and stop at will to not starve the main thread...it's a non-starter to require it.
Oh? I just remember it warning that it does some sort of whole program analysis and generates a ton of unused code if you disable certain optimizations. Anyway, I guess my point was it might not fix the problem for web platforms, but it could still be a nice optional fix for all the others.
I believe asyncify ends up running one's code as an interpreted bytecode
There was an older version around 2016 that did so, but this is not true anymore. I believe it was around 2019 it changed. It will at most slow down the code by 50% - at least in benchmarks. So both are saying things that were/are correct, but slembcke has the more updated info.
There's a blogpost in the docs that has more details.
I might be beating a dead horse here, but I thought I would whip up an example: https://gist.github.com/slembcke/f66773c99c33bced69823a6073d8fcce
This implements an SDL-like main loop API on top of Sokol App, a callback style API. Benefits:
- Everything runs on the main thread (no context switches or thread synchronization)
- Transparent to the user (...mostly)
- Support for ARM and x64 platforms is straightforward
- Easy to implement as optional at runtime
if(fiber != NULL) resume(fiber); else do_normal_thing;etc
Downsides:
- May never work well on the web because of how javascript and WASM work
I'm not familiar with the details, but this should make fixing the "window interactions are blocking on Windows" issue too. If you run the blocking call on a separate fiber, you can yield back to the main loop during the repaint callbacks.
May never work well on the web because of how javascript and WASM work
Emscripten supports fibers via Asyncify. It's quite inefficient compared to a native assembly implementation (performance is comparable to ucontext on Linux and the code size bloats up), but it seems to be doing well enough for the master branch of Taisei, which uses hundreds of coroutines to drive game logic. You certainly would not feel the overhead if you have only one such fiber in your program, at least in terms of runtime performance.
Code size is another matter, but it's possible to cut down on it a lot if you know which code paths should and shouldn't be instrumented. ASYNCIFY_IGNORE_INDIRECT=1 is an easy and huge win if you know that your code will not suspend across indirect function calls, and if you only have a few well known suspension points, you can whitelist their call chains in ASYNCIFY_ONLY and eliminate almost all of size overhead.
Then there is also the wasm stack switching proposal that is supposed to obsolete Asyncify… sometime in the future.
My yet-another stackful coroutine C library has an emscripten fiber backend, for reference.
Exactly. Asyncify comes with a bunch of gotchas like that. To the application developer, restricting yields across indirect calls could be something they design around. On the other hand, it would be really awkard for SDL to say you can't wait for events, swap buffers, or something like that inside an indirect call though. Especially if virtual calls count as indirect. (I assume?) If WASM has the biggest problem with control inversion, and also is the most problematic to apply co-routines to... Eh, I'm not going to be surprised when somebody says no, even if I like solving problems with coroutines. :p
instead of the app having a main loop that collects input, thinks, then renders, they want the apps to provide a callback that fires once per frame, and the platform handles the main loop
This would also be a natural model if people want to plug SDL into a GTK or Qt app: for instance SDL rendering game graphics inside the main content rectangle, but with GTK or Qt chrome (menubar, etc.) around it. The GLib event model used in GTK can be used either way round (integrated into a third-party main loop, or used as the main loop that third-party code integrates into) but the usual way is for it to be in overall control of everything that happens on the main thread, with application code registering callbacks for "do this in about x milliseconds' time", "do this once per frame", "do this whenever you are otherwise idle" or "wake me up when this socket becomes readable". It's been a while since I did Qt, but if I remember correctly it's conceptually quite similar.
I suspect mobile OSs' GUI toolkits also behave like this, because a steady state where literally nothing is happening (with the application blocked in a poll() that will wake up as soon as something interesting happens) is good for power consumption.
BUT...even well defined apps can have That One Function Somewhere That Blocks
In the GLib and Qt worlds, these tend to be somewhere between undesired and forbidden, with callback-based async I/O strongly encouraged for things like networking and D-Bus (as well as the windowing/GUI system). This also makes it much less likely to get into a deadlock where the GUI is no longer updating or responsive. However, I know from experience that nice async APIs for things like D-Bus are hard to do unless you have a more general async framework like GLib's GAsyncResult/GCancellable that an application developer only has to learn once.
On which platforms will the new callbacks be required? Also (hypothetically), will it be possible in the SDL3 to create custom main loops, without using callbacks?
I hope that the API will remain compatible with what we had before, i.e. that it will be possible to implement the main loop yourself, at least on desktop platforms.
I am not a fan of the model, but agree that fighting the inertia here is probably not worthwhile.
There's some advantages to be had from SDL taking over the main loop even on platforms that don't require it
- can make framerate limiting an internal feature, and probably do a better job of it than the average attempt too
- easier to implement async disk I/O when OS threads aren't available - just do smaller reads and writes inbetween callbacks. Not that it's common besides emscripten
- good opportunity to add more asynchronous APIs, especially a main thread task queue (pretty sure emscripten, Cocoa, UIKit, Android, and Win32 all provide their own ways to do this already). This would actually make this inverted control more favorable to me personally, since nearly all of my non-trivial main loop logic is async stuff, I don't know about anyone else though.
- done right, aligns SDL more closely with the Win32 model, fixing the eternal modal loop hiccups there.
I would not want to force this on SDL users, but it might be nice to find a way for apps to be able to adapt cleanly to this way of thinking and/or have the option to work this way.
Having it both ways seems perfectly feasible here, except some targets (like single-threaded emscripten) may not be worth trying to support with the classic model.
I wonder if this is an opportunity to beef up the SDL_main functionality.
I'm going to take a run at this idea and see what happens.
Related to this, I recently red this write-up about re-implementing wipeout. I think the following quote is related to this feature:
Currently it compiles with two different platform backends: SDL2 and Sokol. Both of these support multiple platforms (e.g Windows, macOS, Linux, Android, iOS…). Adding a new platform backend – say, for the Nintendo Switch – is straight forward and (in theory) doesn't necessitate any changes to the game code.
I initially developed with the SDL backend and later added the Sokol libraries. Both are an absolute pleasure to work with.
I was especially impressed with how smooth the compilation for WASM with Sokol went. You compile the whole thing with emcc and it just works. Rendering, input, sound, everything was there. Not a single change in the code needed.
Looks like sokol is callback based.
