SDL2 loses key events/state on viewport creation/destruction

[lailoken]

Version/Branch of Dear ImGui:

1.92.0 WIP (19193)

Back-ends:

imgui_impl_sdl2 + imgui_impl_opengl3

Compiler, OS:

GCC

Full config/build information:

Dear ImGui 1.92.0 WIP (19193)
--------------------------------
sizeof(size_t): 8, sizeof(ImDrawIdx): 4, sizeof(ImDrawVert): 20
define: __cplusplus=202002
define: IMGUI_DISABLE_OBSOLETE_FUNCTIONS
define: __linux__
define: __GNUC__=13
define: IMGUI_HAS_VIEWPORT
define: IMGUI_HAS_DOCK
--------------------------------
io.BackendPlatformName: imgui_impl_sdl2
io.BackendRendererName: imgui_impl_opengl3
io.ConfigFlags: 0x00000481
 NavEnableKeyboard
 DockingEnable
 ViewportsEnable
io.ConfigViewportsNoAutoMerge
io.ConfigViewportsNoDefaultParent
io.ConfigDockingWithShift
io.ConfigNavCaptureKeyboard
io.ConfigInputTextCursorBlink
io.ConfigWindowsResizeFromEdges
io.ConfigMemoryCompactTimer = 60.0
io.BackendFlags: 0x0000140E
 HasMouseCursors
 HasSetMousePos
 PlatformHasViewports
 RendererHasVtxOffset
 RendererHasViewports
--------------------------------
io.Fonts: 12 fonts, Flags: 0x00000004, TexSize: 2048,2048
io.DisplaySize: 1.00,1.00
io.DisplayFramebufferScale: 1.00,1.00
--------------------------------
style.WindowPadding: 10.00,13.00
style.WindowBorderSize: 1.00
style.FramePadding: 6.00,3.00
style.FrameRounding: 0.00
style.FrameBorderSize: 1.00
style.ItemSpacing: 5.00,4.00
style.ItemInnerSpacing: 1.00,2.00

Details:

There seems to be a moment that key events and/or down state is lost when viewports are created / destroyed in SDL.

For instance, in the example below, when I hold down "O" the window flickers in and out correctly on each frame. But sometimes (I think mostly when the window is closed) when I release "O", the release was not registered and thus "sticks".

I have confirmed in the ImGui Metrics editor that the key is indeed registering as down.

There is a similar issue when I hold in Ctrl to show a popup tooltip, and when this tooltip extends outside the window boundary, causing a new viewport to be created, I seem to lose a frame of ctrl being held down, and my tooltip starts to flicker.

This happens in SDL on X11 in Ubuntu, and have confirmed this on two different user's workstations.

Screenshots/Video:

No response

Minimal, Complete and Verifiable Example code:

void test()
{
    static bool renderExtraWindow = false;

    if (ImGui::Begin("Main Window", nullptr, LocalConfig::ViewportWindowFlags()))
    {
        if (ImGui::Button("Render Extra Window"))
            renderExtraWindow = !renderExtraWindow;
        if (ImGui::IsKeyDown(ImGuiKey_O))
            renderExtraWindow = !renderExtraWindow;
    }
    ImGui::End();

    if (renderExtraWindow)
    {
        if (ImGui::Begin("Extra Window", &renderExtraWindow, LocalConfig::ViewportWindowFlags()))
        {
            ImGui::Text("Extra Window");
        }
        ImGui::End();
    }
}

[ocornut]

I would guess that a destroyed window doesn't receive a key up event. For GLFW we have workaround for this (original commit was a4adf6057677c780907b7fc38e2695199bccd446 it looks a bit different today but essentially is same logic). I guess we could try implementing that in other backends.

[ocornut]

There is a similar issue when I hold in Ctrl to show a popup tooltip, and when this tooltip extends outside the window boundary, causing a new viewport to be created, I seem to lose a frame of ctrl being held down, and my tooltip starts to flicker.

That's actually a different issue for which the suggested workaround above wouldn't work.

For both issues you may need to investigate since I cannot seem to reproduce on Windows.

Uncommenting the IMGUI_DEBUG_LOG calls in SDL_KEYDOWN/SDL_KEYUP handler + adding extra IMGUI_DEBUG_LOG in ImGui_ImplSDL2_CreateWindow() and ImGui_ImplSDL2_DestroyWindow() would likely provide raw data to act upon (exactly understand how events are fired or not fired on creation+focus loss and destruction). Note that keymods are fed via a specific field of SDL key events and may have different issues than other keys.

It would also be good to see if that repro using the SDL3 example under X11.

[ocornut]

So we are specifically talking about the second issue here:

There is a similar issue when I hold in Ctrl to show a popup tooltip, and when this tooltip extends outside the window boundary, causing a new viewport to be created, I seem to lose a frame of ctrl being held down, and my tooltip starts to flicker.

I'm getting keyup events when I'm not releasing Ctrl. My guess is that in the moments between creation of viewports, SDL creates a frame where my ImGui app does not have any window focused.

Could you modify the handler to be:

case SDL_KEYDOWN:
case SDL_KEYUP:
{
    ImGuiViewport* viewport = ImGui_ImplSDL2_GetViewportForWindowID(event->key.windowID);
    IMGUI_DEBUG_LOG("SDL_KEY%s : key=0x%08X ('%s'), scancode=%d ('%s'), mod=%X, windowID=%d, viewport=%08X\n",
        (event->type == SDL_KEYDOWN) ? "DOWN" : "UP  ", event->key.keysym.sym, SDL_GetKeyName(event->key.keysym.sym), event->key.keysym.scancode, SDL_GetScancodeName(event->key.keysym.scancode), event->key.keysym.mod,
        event->key.windowID, viewport ? viewport->ID : 0);
    ImGui_ImplSDL2_UpdateKeyModifiers((SDL_Keymod)event->key.keysym.mod);
    if (viewport == nullptr)
        return false;
    ImGuiKey key = ImGui_ImplSDL2_KeyEventToImGuiKey(event->key.keysym.sym, event->key.keysym.scancode);
    io.AddKeyEvent(key, (event->type == SDL_KEYDOWN));
    io.SetKeyEventNativeData(key, event->key.keysym.sym, event->key.keysym.scancode, event->key.keysym.scancode); // To support legacy indexing (<1.87 user code). Legacy backend uses SDLK_*** as indices to IsKeyXXX() functions.
    return true;
}

I would like to confirm:

• if all values of event->key.windowID are valid and correspond to an existing viewport.
• which viewports are associated to each event of the UP/DOWN sequence.

We could decide to ignore UP event coming on a destroyed viewport.

But it would be good if you can also repro without the "flicker", aka show the tooltip regardless of Ctrl being held, and confirm the KEYUP/KEYDOWN sequence without the flickering back and worth. This is mostly because after the first occurrence of the issue, we have consecutive "delete viewport / add viewport" with same viewport ID, this why I also included SDL window ID in the log.

[lailoken]

I've spent quite a bit of time testing and trying, and I find the following solves all my problems (as far as I can tell). (I think SDL reqally requires these properties...)

The implementation may be made better and more robust with better integration into ImGui window flags:

This needs to be modified in ImGui_ImplSDL2_CreateWindow():

#if !defined(_WIN32)

    // ADDED: Our *best* heuristic to determine if a viewport is a tooltip or popup menu:
    if (viewport->Flags & (ImGuiViewportFlags_TopMost | ImGuiViewportFlags_NoTaskBarIcon | ImGuiViewportFlags_NoFocusOnClick | ImGuiViewportFlags_NoFocusOnAppearing |
                           ImGuiViewportFlags_NoDecoration | ImGuiViewportFlags_NoTaskBarIcon))
    {
        // we do have normal windows that float topmost, so we need to add more checks to see if it's a tooltip or popup menu:
        if (viewport->Flags & ImGuiViewportFlags_NoInputs)
        {
            // IMGUI_DEBUG_LOG("[viewport] Viewport %p is a tooltip\n", viewport);
            sdl_flags |= SDL_WINDOW_TOOLTIP;  // this is required else there is a input focus stealing blip.
        }
        else if (viewport->Flags & ImGuiViewportFlags_NoFocusOnClick)
        {
            // IMGUI_DEBUG_LOG("[viewport] Viewport %p is a popup menu\n", viewport);
            sdl_flags |= SDL_WINDOW_POPUP_MENU;  // this is required else menus are not always above floating topmost normal windows.
        }
    }

    // See SDL hack in ImGui_ImplSDL2_ShowWindow().
    sdl_flags |= (viewport->Flags & ImGuiViewportFlags_NoTaskBarIcon) ? SDL_WINDOW_SKIP_TASKBAR : 0;
#endif

[lailoken]

PS: Our app has a floating viewport app bar with a menu, and I find that on SDL I have to disable its topmost property else its own menus are obscured by it. Windows DX11 is working fine here.

I really think that these SDL window attributes are needed for tooltips / menus. But I'm not always sure when something is a "tooltip" (modal popups?)

[ocornut]

You can cast ImGuiViewport* to ImGuiViewportP* and access the window directly to check for tooltip/popups flags, but I'd be happy to formalize this in an official manner. Along with #7950 this might be one of the only first time since 2018 (#2117) that someones make a constructive push toward fixing X11 issues....

We should establish many things with more clarity however:

• Exactly what SDL_WINDOW_TOOLTIP and SDL_WINDOW_POPUP_MENU do on every system.
• If e.g. SDL_WINDOW_TOOLTIP meant "do not take focus" then it should be set on ImGuiViewportFlags_NoFocusOnAppearing and ImGuiViewportFlags_NoInputs ? But it doesn't seem to mean that.
• Does your aforementioned change works on Windows for you?

PS: Our app has a floating viewport app bar with a menu, and I find that on SDL I have to disable its topmost property else its own menus are obscured by it. Windows DX11 is working fine here.

• TopMost is only set for tooltips, so I don't understand how/why your floating app-bar would be affected?

I think I'd be ready to add this to the codebase now:

        if (viewport->Flags & ImGuiViewportFlags_NoInputs)
            sdl_flags |= SDL_WINDOW_TOOLTIP;  // this is required else there is a input focus stealing blip.

Since this is what it does. But in order to better honor ImGuiViewportFlags_NoFocusOnAppearing it may be worth investigating the keyup/keydown events and their originating window.

SDL3 says:

* - `SDL_WINDOW_TOOLTIP`: window should be treated as a tooltip and does not
*   get mouse or keyboard focus, requires a parent window
* - `SDL_WINDOW_POPUP_MENU`: window should be treated as a popup menu,
*   requires a parent window

* - `SDL_WINDOW_TOOLTIP`: The popup window is a tooltip and will not pass any
*   input events.
* - `SDL_WINDOW_POPUP_MENU`: The popup window is a popup menu. The topmost
*   popup menu will implicitly gain the keyboard focus.

---

Exactly what SDL_WINDOW_TOOLTIP and SDL_WINDOW_POPUP_MENU do on every system.

SDL3, Win32, SDL_WINDOW_TOOLTIP:

• make input transparent (return HTTRANSPARENT in WM_NCHITTEST msg).

SDL3, Win32, SDL_WINDOW_POPUP_MENU:

• forward focus when showing window and parent has focus, and vice-versa when hiding.

[lailoken]

The current docking branch is not working correctly.

When I do the following:

void test()
{
    ImGui::Begin("Test", nullptr, ImGuiWindowFlags_NoInputs);
    ImGui::Text("Test");
    ImGui::End();
}

This used to have a (non-floating) window without inputs.

Now it's also topmost-floating.

Also, the same code in SDL3 does not show any window when the ImGuiWindowFlags_NoInputs flag is added.

So in SDL2 SDL_WINDOW_TOOLTIP also makes it topmost (above even other topmost (floating) windows).

For now my code is working better, but I wish there was a reliable way to know when a viewport was a menu vs a popup/tooltip type thing.

[lailoken]

I've been running that fix in ImGui_ImplSDL2_CreateWindow for some time now with great success, but I recently ran into a problem relating to how some docked windows behave.

It now looks like this:

    if (viewport->Flags & ImGuiViewportFlags_NoFocusOnClick && viewport->Flags & ImGuiViewportFlags_NoFocusOnAppearing && viewport->Flags & ImGuiViewportFlags_NoDecoration &&
        viewport->Flags & ImGuiViewportFlags_NoTaskBarIcon)
    {
        // we do have normal windows that float topmost, so we need to add more checks to see if it's a tooltip or popup menu:
        if (viewport->Flags & ImGuiViewportFlags_NoInputs && viewport->Flags & ImGuiViewportFlags_TopMost)
        {
            // IMGUI_DEBUG_LOG("[viewport] Viewport %p is a tooltip\n", viewport);
            sdl_flags |= SDL_WINDOW_TOOLTIP;  // this is required else there is a input focus stealing blip.
        }
        else
        {
            // IMGUI_DEBUG_LOG("[viewport] Viewport %p is a popup menu\n", viewport);
            sdl_flags |= SDL_WINDOW_POPUP_MENU;  // this is required else menus are not always above floating topmost normal windows.
        }
    }

(The issue I had is that when I create a window initially docked, and dragged it out, it gets created outside the dock with ImGuiViewportFlags_NoInputs, but it is created initially undocked, that flag is not set... not sure if this is a intended?)

These changes work more consistently for me (at least for all my windows)

[ocornut]

The issue I had is that when I create a window initially docked, and dragged it out, it gets created outside the dock with ImGuiViewportFlags_NoInputs, but it is created initially undocked, that flag is not set... not sure if this is a intended?

While moving a viewport in a situation where it could be docked, we temporarily set ImGuiViewportFlags_NoInputs on the viewport in order to be able to query the platform OS window under it. We clear the flag when not moving the viewport anymore.

Details: (maybe not useful to your question)

• We rely on backend supporting ImGuiBackendFlags_HasMouseHoveredViewport to report the viewport hovered by mouse by calling io.AddMouseViewportEvent().
• When the backend does not support ImGuiBackendFlags_HasMouseHoveredViewport, or, when we detect they called io.AddMouseViewportEvent() with a viewport that currently has the ImGuiViewportFlags_NoInputs flag, we use our own heuristic to infer platform window Z order from the last time a platform was focused. This is not perfect and among other things, won't notice if there's a foreign app window between 2 imgui viewports managed by the app.