Flickering when resizing issue

[deprilula28]

Describe the bug

When resizing, win32 expects that a new frame will be presented to the swapchain before returning from the resizing event callback, otherwise it will clear the window to white, leaving a flicker until the next frame gets rendered.

As a mitigation, example 02 with resizing will use a conditional variable to wait until the next frame is rendered within the callback:

https://github.com/Devsh-Graphics-Programming/Nabla-Examples-and-Tests/blob/f105defe618f1630665f5a4c632c78fd63b5d323/02.ComputeShader/main.cpp#L202

https://github.com/Devsh-Graphics-Programming/Nabla-Examples-and-Tests/blob/f105defe618f1630665f5a4c632c78fd63b5d323/02.ComputeShader/main.cpp#L556

However this is not scalable with anything that takes longer to render. A better solution is to blit the previous presented to the new swapchain, and then it will have the new resized image on the next frame.

We can also look into other mitigations to avoid Windows from clearing our window, such as WM_PAINT as mentioned here:

https://www.reddit.com/r/vulkan/comments/fg8lg8/comment/fk3cbw4/?utm_source=share&utm_medium=web2x&context=3

https://stackoverflow.com/a/58891586/5572963

Steps to Reproduce

• [ ] Try running the example 02 with resizing without the conditional variable code mentioned above.

[devshgraphicsprogramming]

~~Actually I think the bug is caused by forgetting that a window resize event is processed (and executed by the OS) after onWindowResized_impl returns control (as the return value tells us whether a resize/minimize/maximize/move is allowed to happen).~~

~~Therefore you were recreating swapchains and their resources, but just before the resize (which puts them out of date)~~

Actually Win32 API works different to X11 which is what our callback system was modelled on.

[deprilula28]

The crux of the problem:

• Windows resize event happens after the window has already resized, returning false in the callback (not calling DefWindowProc) does not stop the resize
• Resize event is triggered during a vk present on the old swapchain (!)
  • Vulkan says that contents and usages of already acquired images are valid, but only if they're already acquired by the time the swapchain goes "out of date" and before they're presented

After oldSwapchain is retired, the application can pass to vkQueuePresentKHR any images it had already acquired from oldSwapchain. E.g., an application may present an image from the old swapchain before an image from the new swapchain is ready to be presented. As usual, vkQueuePresentKHR may fail if oldSwapchain has entered a state that causes VK_ERROR_OUT_OF_DATE_KHR to be returned. https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VkSwapchainCreateInfoKHR.html

• Since Windows likes to trigger the event callback exactly during the present this means you don't have a single image from the old swapchain thats valid to use and has defined contents.

What this means:

• For a slow rendering app ("Heavy offline Industrial Rendering") we need to "triple buffer", so we always have an image we can blit from into the new swapchain on resize
• For a realtime app, we can just recreate and wait until the main thread catches up before the next V-Blank

My TODO:

• onResize callback

std::unique_lock lk = IGraphicalApplication:recreateSwapchain(w,h,m_swapchain); // locks `IGraphicalApplication::m_swapchainPointerMutex`
#ifdef NO_FLICKER
// m_presentedResourceIx would be set to `m_resourceIx` inbetween the submit that dispatches the work which will produce `m_tripledBufferImage` contents and the present 
CommonAPI::waitForFrame(m_frameComplete[m_presentedResourceIx]); // same stuff as e52cd04548bb0090639a4ab327b2bf3e8de99ff7/02.ComputeShader/main.cpp#L439-L453
// intentionally no reset
// acquires image, allocates one shot fences, commandpool and commandbuffer to do a blit, submits and presents
IGraphicalApplication::immediateImagePresent(m_queue,m_swapchain,m_tripleBufferImage[m_presentedResourceIx]);
#endif

• Main thread

// make sure the swapchain pointer we use does not change suddenly during the frame
uint32_t imageIx;
core::smart_refctd_ptr<ISwapchain> swapchain = IGraphicalApplication::acquire();
//! which would do the following
{
   std::unique_lock lk(m_swapchainPointerMutex);
  CommonAPI::waitForFrame(m_frameComplete[m_resourceIx]); // same stuff as e52cd04548bb0090639a4ab327b2bf3e8de99ff7/02.ComputeShader/main.cpp#L439-L453
  m_frameComplete[m_resourceIx]->reset();
   swapchain = m_swapchain;
   // swapchain cannot get retired before we acquire, so mutex
   swapchain->acquire(imageIx);
}

auto& cb = m_cmdbuf[m_resourceIx];

... heavy rendering stuff ...

// make sure special resize-blit happens before or after `m_presentedResourceIx` is updated
{
  std::unique_lock lk(m_swapchainPointerMutex);
  m_presentedResourceIx = m_resourceIx;
}
CommonAPI::present(swapchain,imageIx);

[devshgraphicsprogramming]

New Example 08 Hello Swapchain does this properly.

The only improvement it could see (and any app that expects sub 30 FPS rendering) is to pick a different queue from the main Graphics Queue for acquire-blit-present.