libsdl-org
Frame Lag on Metal SDL3 GPU
You have to use 'CVDisplayLink' on macos, record your command buffer there and call [commandBuffer presentDrawable:view.currentDrawable atTime: outputTime]; using the 'const CVTimeStamp* outputTime' from the CVDisplayLink argument. Otherwise you get very noticeable frame lag especially when resizing imgui windows.
https://github.com/user-attachments/assets/796218a0-0490-4b8f-84ac-d82e26b8065b
Assigned this to @TheSpydog who handles the Metal stuff, but it sounds like the solution already exists... we can review a patch for this if so!
I'll need to do some more testing, few apps get this right. chrome gets it, if you go to https://www.w3schools.com/howto/howto_js_draggable.asp you can see it follows your mouse, unlike blender where if you drag the hello cube or resize the side panel, it doesn't follow your mouse
damn, how do you add symbols to the sdl api? I can modify the existing functions but I can't add new symbols for the life of me, it's not linking when I use it in my main function
The easiest way (in my experience, at least) is to add the symbol to the header and a C/M file as you normally would, then you can run src/dynapi/gendynapi.py and it'll pick up the new export and make all the scary files Just Work.
https://github.com/user-attachments/assets/f56ca65b-e7fe-441d-a619-8dc7ab3cc7df
yes! Finally got it to work damn. Full vsync but without the lag, visual tearing or stuttering
I've know about it for years but it'd never work for me properly and that's what really pissed me off about bgfx was theirs was doubly slow for some reason, and CVDisplayLink would only slightly work with vsync and I thought that was the most you can get out of it. Turning off vsync (and still running at 60 fps since CVDisplayLink fires at monitor rate) would follow the mouse but cause intense stuttering and tearing.
https://github.com/user-attachments/assets/c83275aa-e117-4ba2-afeb-2980917a5bf3
I got fed up with bgfx because of the ridiculous sluggishness and switched to opengl and native input sampling and it was perfect, but when I wanted to switch to metal, it wasn't perfect, and then I accidentally discovered opengl was perfect only because it was running at 400 fps. Afterwards I got obsessed with this again since chrome somehow does it perfectly, it must be possible. Chatgpt told me about passing the output time to the CAMetalLayer draw overload, then I had an epiphany, the issue all along was that rendering at 60 caused vsync to step in, also CVDisplayLink expects you to render the frame before returning kCVReturnSuccess I think, it works better this way then just toggling a semaphore to reactivate the main thread. The issue is it runs on a different high propriety thread so you have to ping pong back to the main thread (btw, is there any way in sdl3 to change threads to allow this thread to be the draw thread and call SDL_SubmitGPUCommandBuffer from it?). Anyway dropping 1 frame every 60 got it to work today, I guess if you figure out how to time it properly or at 59.99fps or not enqueue a frame if the previous one hasn't been shown, vsync wouldn't step in, also making draw calls exclusively from the CVDisplayLink Thread without toggling a main thread semaphore, then waiting for main thread to return back to you. Then it'd be really perfect
Ok I was wrong, the entire point of CVDisplayLink is to get the output time of the frame, the only thing the function should do is notify your main thread, give it the time and exit. The callback fires 1.5 frames in advance, you subtract 1 frame's time and subtract how long you think your logic would take. The entire reason it works or is needed is because on macos vsync does no accounting whatsoever for your logic, so straight after finishing a frame, you spend 0.001 seconds sampling input & logic and then presentDrawable forces you to wait 0.015666 for the next vblank to present that by now old data
https://github.com/user-attachments/assets/55d163c4-890c-4abf-bf2c-f7645e8a9d80
vsync also tends to get in the way of your CVDisplayLink, you can try dropping frames if you suspect there are queued unflushed frames (there is no api to detect this). you can turn off vsync and it works but if you underestimate the time you'll need for your logic, and try to present the frame late you'll get intense tearing and stuttering
ok, I think I get it now. fixed the issue where it wasn't working after minimising / turning off the link and its pr worthy
https://github.com/user-attachments/assets/de6ccb30-fbf0-4f09-b516-90350b50155d
https://github.com/user-attachments/assets/6364931e-7aec-4ae3-8aa4-6cf41d3e88a2 I'll try to make a pr tomorrow, need to implement non blocking AcquireGPUSwapchainTexture, and it doesn't do anything silly anymore like skip 1 frame in 60 or require new function calls. turns out to be a lot simpler than I thought. Still I'm incredibly surprised neither bgfx nor the sdl renderer nor sdl3 gpu or basically 90% of programs implement this
you can check out this minimal imgui example and feel how smooth it is. https://github.com/alexgu754/LowLatencyOSX