Investigation into window resizing

[deprilula28]

Description

What do we need to change for window resizing, how do the different surface implementations report & deal with resizing.

Programatic Resizing APIs

Resize the window to a given size (not the same as the OS telling us the user has resized).

Windows

For Windows, SetWindowPos with SWP_NOMOVE and SWP_NOREPOSITION may be used. By default, this will also "activate" the window, which can be disabled with SWP_NOACTIVATE, and it will change the Z order of the window, which can be controlled with hWndInsertAfter or disabled with SWP_NOZORDER.

Normally this has to be called from the window thread, but the SWP_ASYNCWINDOWPOS flag makes it thread safe.

SetWindowPos(
    hwnd,
    // Unused due to flags
    nullptr, 0, 0,
    width, height,
    SWP_NOMOVE | SWP_NOREPOSITION | SWP_NOACTIVATE | SWP_NOZORDER | SWP_ASYNCWINDOWPOS
);

X11

For X11, XResizeWindow may be used.

x11.pXResizeWindow(m_dpy, m_native, width, height);

Normally X11 requires XInitThreads to be the first call to xlib in order for its usage to be thread safe. This comment, however, indicates that EGL handles the per-display locking & unlocking for us (?). - Locking and unlocking with the X11 API (XLockDisplay & XUnlockDisplay) requires having called XInitThreads. - Will EGL cover for us in the resize case?

Wayland

It appears the intended function is wl_egl_window_resize, under Wayland EGL. It appears this function is not thread safe. According to this, the last two arguments are related to moving floating top-level windows (?). We can leave them as 0.

wl_egl_window_resize(m_native, width, height, 0, 0);

Mac OS

For Mac, setContentSize within NSWindow may be used. This function does not appear to be thread safe, and needs to be called from the window thread.

ui::IWindow API

• Provide programatic set window size function:

    virtual void setWindowSize(uint32_t width, uint32_t height) = 0;

• Handling thread safety
  • We need to able to communicate with the window thread for setting the window size.
    • Using a mutex is not enough in the case of Mac OS or Wayland, it needs to be done from the window thread.
    • Workaround for Wayland used in egl-wayland: https://github.com/NVIDIA/egl-wayland/pull/53/files
    • (Not needed on Win32 with SWP_ASYNCWINDOWPOS or X11 with XInitThreads or EGL locking (?) )

Resizing in Vulkan

Resizing plan:

• Create new swapchain providing oldSwapchain in VkSwapchainCreateInfoKHR.
• Use a mutex to have locks within the swapchain recreation and swapchain image acquisition on render loop.
• ISwapchain should keep track of an "iteration" value that increases each time the swapchain is re-created.
  • Implementation keeps track of the last swapchain iteration used for each frame in flight.
    • Get the image, re-create the image view and FBO for current frame in flight if the swapchain superceeds it.
      • This takes advantage that old swapchain images remain valid until a new view is created with the new swapchain. So we can avoid waiting for idle and still render to the old swapchain for all the frames in flight, only creating a new one for the current m_resourceIx.
  • With the current API for createSwapchain, vkGetSwapchainImagesKHR is called right away, meaning all of the old images get invalidated.
    • Would need something like getImage(uint32_t index) on ISwapchain.

Platform specific notes

VK_KHR_win32_surface

• Interesting note here:

Creating a VkSwapchainKHR over a window object can alter the object for its remaining lifetime. Either of the above alterations may occur as a side effect of vkCreateSwapchainKHR.

Old Experiment

• Makeshift solution for resizing:

WIN_W = window->getWidth();
WIN_H = window->getHeight();

// Destroy old swapchain
const video::CVulkanLogicalDevice* vkDevice = static_cast<const video::CVulkanLogicalDevice*>(device.get());
const video::CVulkanSwapchain* vkSwapchain = static_cast<const video::CVulkanSwapchain*>(swapchain.get());

VkDevice vk_device = vkDevice->getInternalObject();
auto* vk = vkDevice->getFunctionTable();

vk->vk.vkDestroySwapchainKHR(vk_device, vkSwapchain->getInternalObject(), nullptr);

// Create new swapchain
createSwapchain();

• Resizing on the resize event

  • In format of the examples, the render loop continues as the resize event is being handled, causing a segfault on the present function.

• Resizing on the main render loop shows a white screen for a few frames after resizing (flickering).

  • Device wait for idle doesn't fix this.
  • This also tells us that the old swapchain turns white and stops presenting after the resize event, at least on Windows, so we can't use the old swapchain during a resize or something similar.

API for handling resizing

• Resizing the swapchain
  • Provide oldSwapchain in VkSwapchainCreateInfoKHR.
  • Old images acquired from the old swapchain may still be used until the new swapchain's images are acquired.
  • Use a mutex/other lock to ensure presenting/acquiring the image doesn't run at the same time as the recreation.

~~- The biggest issue is the render loop running concurrently with the window events, not allowing us to resize within the window event - We could have the rendering be done within the event handler, something like a redraw callback. - Make the event loop non-blocking, if there is no event, redraw - Redraw events? Seems to be WM_PAINT on Windows - Current event loop doesn't trigger this

• Recreate & handle automatically on swapchain->acquireNextImage?

  • acquireNextImage returns "suboptimal" when the sizes don't match
  • A simple check for the sizes could also suffice, but would need to be an input into acquireNextImage
  • As tested above, this causes flicker.~~

• Views to the swapchain images & their FBOs

  • These need to be replaced with the new swapchain images before we use them again