Bug and Possible Fix: Judder on Window Resize (MacOS with Metal)

[jceipek]

This is easiest to see with the imgui-metal sample: when you resize the OS window from the right side, the contents of the window exhibit an ugly "judder" effect.

This appears to be the same issue discussed here: https://thume.ca/2019/06/19/glitchless-metal-window-resizing/

What’s happening is often the new Metal frame doesn’t arrive in time and it draws a stretched version of the previous frame instead.

Tristan Hume provides a test program https://github.com/trishume/MetalTest that seems to fix the issue (written in Swift but it's pretty easy to port to Objective C). Note that it doesn't include any animation -- all of the rendering is explicit rather than happening on a timer or being synced to the display refresh rate, which might be fine for a purely event driven application, but not for one that does smooth animation.

As far as I understand it, MTKView, which sokol_app.h appears to be using, provides display-synced updates automatically, but it appears that it may not be configurable enough to fix the issue -- you may have to use CAMetalLayer directly in a custom subclass of NSView that implements the CALayerDelegate protocol. There's an Apple sample app here https://developer.apple.com/documentation/metal/drawable_objects/creating_a_custom_metal_view?language=objc that demonstrates how to implement a display-synced render loop using CVDisplayLink. Unfortunately, I haven't personally succeeded in modifying this demo to fix the resize judder issue (although I could have easily missed something).

I've only just started learning how to use explicit graphics APIs like Metal (I've mostly used Unity for my graphics projects so far) and would love to know if you have any insight into how to fix this issue properly.

[floooh]

Thanks a lot for researching this issue! TBH I'm not sure if it's worth fixing though if it means giving up the convenience of MTKView and using the lower level components directly :)

If anything, I'd like to fix the resizing behaviour on Windows first, because this blocks rendering for the entire resizing period, so it's much worse than a bit of judder ;)

I'll keep the bug open as a reminder, because it might be worth doing after all in the future.

[jceipek]

That makes a lot of sense! The Windows behavior sounds rough.

I played around a bit more with various components and still couldn't really figure out how to get Apple's custom metal view demo not to glitch -- I don't understand enough about the interplay between presentsWithTransaction, displaySyncEnabled, layerContentsRedrawPolicy, allowsNextDrawableTimeout, needsDisplayOnBoundsChange, waitUntilScheduled, commit, and present to understand what the problem is there.

It might be worth mentioning that setting layerContentsPlacement to NSViewLayerContentsPlacementTopLeft seems to mask the problem to the degree that it is barely noticeable when rendering a UI, but I haven't had a chance to try to get that working on top of MTKView yet.

I'll probably come back to exploring proper window initialization some time in the future and will report back if I figure something out :)

[floooh]

This has recently been fixed via https://github.com/floooh/sokol/pull/963