Retaining focus on InputText with on-screen keyboard

[masranber]

Version/Branch of Dear ImGui:

Version: 1.89 WIP Branch: master

Back-end/Renderer/Compiler/OS

Back-ends: imgui_impl_opengl3.cpp w/ custom windowing/IO backend Compiler: g++ Operating System: Ubuntu 22.04 LTS

My Issue/Question:

I'm working on making an on-screen keyboard for text input into my UI because a physical keyboard is not available for my use case.

Per the FAQ, I'm currently using the io.WantTextInput flag to know when to draw/hide the on-screen keyboard window. Each key on the keyboard is a button that calls io.AddInputCharacter() with its corresponding char when clicked. At some point, I'll need to add different io handling for backspace, shift, etc... keys but that's irrelevant for this discussion.

The issue I'm running into is the currently focused InputText will immediately lose focus when the user interacts with the on-screen keyboard window. This is obviously the designed behavior. However, once the InputText loses focus, io.AddInputCharacter() (or any key press event) won't be registered with the InputText. Furthermore, io.WantTextInput gets set to false when the InputText loses focus so the on-screen keyboard closes. So I need a way to overcome the InputText losing focus.

I've attempted to fix this by saving and restoring the current focus before and after the on-screen keyboard is drawn. Since this didn't appear to be possible via imgui's public API, I resorted to internal API in imgui_internal.h.

This attempted fix works as follows:

// Draw other windows, including InputTexts

auto io = ImGui::GetIO();
if(io.WantTextInput) {
    // Save focus
    ImGuiID focused = ImGui::GetFocusID(); // imgui_internal.h, should focused widget be the InputText wanting input????
    ImGuiWindow *focusedWindow = ImGui::GetCurrentWindow(); // doesn't seem to return the focused window, GetFocusedWindow() doesn't exist

    // Draw on-screen keyboard window, handle key press events

    // Restore focus
    if(focused && focusedWindow) ImGui::SetFocusID(focused, focusedWindow); // imgui_internal.h, restore focus to InputText
}

However, this isn't working. No text gets inserted into the InputText and focus isn't restored after this block of code runs. I've tried referencing #1418 and #718, but I don't think the SetKeyboardFocusHere() API accomplishes what I need when the InputText and keyboard drawing are separated. Push/PopAllowKeyboardFocus() also seems to have no effect on the focus behavior.

I see to be left with 3 potential solutions here:

1. Figure out how to save/restore focus to the InputText. Am I doing something wrong in my example??
2. Prevent the on-screen keyboard window from stealing focus. This probably requires modifying imgui's core behavior for windows and buttons.
3. Run the on-screen keyboard in a separate imgui context, storing a pointer to the primary context's ImGuiIO instance. However I'm not entirely sure how well the GLFW/OpenGL backends work with multiple contexts. I feel this will turn into a rabbit hole and I'd like to rule out the first two before considering. UNLESS someone can confirm multiple contexts would be supported with minimal effort.

Standalone, minimal, complete and verifiable example: (see https://github.com/ocornut/imgui/issues/2261)

ImGuiWindowFlags windowFlags =
        ImGuiWindowFlags_AlwaysAutoResize |
        ImGuiWindowFlags_NoResize;

static char string1[100];
static char string2[100];

bool ignore1;
if(ImGui::Begin("Window 1", &ignore1, windowFlags)) {
    ImGui::InputText("##input1", string1, IM_ARRAYSIZE(string1));
}
ImGui::End();

bool ignore2;
if(ImGui::Begin("Window 2", &ignore2, windowFlags)) {
    ImGui::InputText("##input2", string2, IM_ARRAYSIZE(string2));
}
ImGui::End();

auto io = ImGui::GetIO();
if(io.WantTextInput) {
    // Save focus
    ImGuiID focused = ImGui::GetFocusID(); // imgui_internal.h, should focused widget be the InputText wanting input????
    ImGuiWindow *focusedWindow = ImGui::GetCurrentWindow(); // doesn't seem to return focused window, GetFocusedWindow() doesn't exist

    bool ignore3;
    if(ImGui::Begin("On-Screen Keyboard", &ignore3, windowFlags | ImGuiWindowFlags_NoFocusOnAppearing)) {
        // just one key for the sake of example
        // Clicking this button or anywhere in the keyboard window will steal focus from the InputText
        if(ImGui::Button("a##osk-key-a")) {
            io.AddInputCharacter('a');
        }
    }
    ImGui::End();

    // Restore focus
    if(focused && focusedWindow) ImGui::SetFocusID(focused, focusedWindow); // imgui_internal.h, restore focus to InputText
}

[PathogenDavid]

I don't have an immediate solution for you, but I can answer some of your questions.

(Also I'm pretty sure there's another on-screen keyboard thread besides https://github.com/ocornut/imgui/issues/1418 but I can't seem to find it.)

Figure out how to save/restore focus to the InputText. Am I doing something wrong in my example??

Dear ImGui has two different concepts you can call focus. There's focused items and active items. The focused item is used by navigation. The active item is used to determine which widget receives input.

You're controlling the focused item, but the active item is what you actually care about. (As noted by this comment, ImGui::SetKeyboardFocusHere is somewhat misleadingly named as it's indirectly controlling the active item.)

Unfortunately this won't be as simple as just using GetActiveID/ActiveIdWindow/SetActiveID instead as that upsets the internals of ImGui::ButtonBehavior. (I didn't have time to look into why.)

Even if that worked, I think you'd likely be losing out on button repeat behavior which seems desirable for an on-screen keyboard.

// doesn't seem to return the focused window, GetFocusedWindow() doesn't exist

You're looking for ImGui::GetCurrentContext()->NavWindow

However I'm not entirely sure how well the GLFW/OpenGL backends work with multiple contexts.

The GLFW backend is known to be problematic when used with multiple contexts.

[ocornut]

Intuitively I feel like using a second context may be the easiest solution. The issue for GLFW backend we can look into fixing - it is a fairly simple change, if just only noisy. OP suggested they have their own windowing backend in the template, but refer to GLFW later on so unsure if they would be directly affected. ~~Most other backends don't have this issue which can be handled on app side.~~ EDIT: SDL backend has same issue, could likewise be fixed.

[masranber]

Thank you for the quick responses.

@PathogenDavid Thank you for clearing up active vs focus. Sounds like the approach I'm trying won't work regardless of whether I'm using the focused or active ID.

@ocornut Since you think the 2nd context may be the easiest solution, I'll focus my efforts there.

To clear up the windowing backend, the main application is using GLFW on Linux. However, imgui has no knowledge of the actual backend because the UI is behind its own interface for platform, compatibility, and app architecture/design reasons. So all the display and input event IO binding between the interface and imgui is custom. Hopefully I can use that to my advantage with multiple contexts.

I'll give the two context approach a shot and update later with my findings. Thanks guys.

[ocornut]

Note that the GLFW/SDL backends will likely need the multi-context fix, even though there's a slight chance it might work without.

[masranber]

I got the on-screen/virtual keyboard working with multiple contexts with imgui_impl_opengl3.cpp. Wasn't too hard. Biggest challenge is making sure the correct context is active at all times, which I added my own context switching API for.

The UI uses its own ImGui IO bindings, not one of the included implementations (GLFW, etc..).

Also this code may not be 100% functional as-is since it's been stripped down.

Context Switching

The purpose of adding proper context switching rather than just calling SetCurrentContext() is to ensure the previous context is restored. Without it, it's easy to start rendering elements in the wrong context, sending IO events to the wrong context, etc... especially once you start mixing in nested context switches.

I also added an RAII wrapper to automatically manage the context switch in the current scope (less verbose and bug prone than doing it manually with Push/PopContextSwitch).

imgui_context_extensions.h

namespace ImGui {

void PushContextSwitch(ImGuiContext *newCtx);
void PopContextSwitch();

class ScopedContextSwitch {
public:
    explicit ScopedContextSwitch(ImGuiContext *newCtx);
    ScopedContextSwitch(const ScopedContextSwitch& copy) = delete;
    ScopedContextSwitch& operator=(const ScopedContextSwitch& copy) = delete;
    ScopedContextSwitch(ScopedContextSwitch&& move) = delete;
    ScopedContextSwitch& operator=(ScopedContextSwitch&& move) = delete;
    ~ScopedContextSwitch();
};

} // namespace ImGui

imgui_context_extensions.cpp

static std::stack<ImGuiContext*> CONTEXT_STACK;

namespace ImGui {

void PushContextSwitch(ImGuiContext *newCtx) {
    CONTEXT_STACK.push(ImGui::GetCurrentContext()); // save current context
    ImGui::SetCurrentContext(newCtx); // perform context switch
}

void PopContextSwitch() {
    if(CONTEXT_STACK.empty()) return;
    ImGui::SetCurrentContext(CONTEXT_STACK.top()); // restore previous context
    CONTEXT_STACK.pop();
}

ScopedContextSwitch::ScopedContextSwitch(ImGuiContext *newCtx) {
    ImGui::PushContextSwitch(newCtx);
}

ScopedContextSwitch::~ScopedContextSwitch() {
    ImGui::PopContextSwitch();
}

} // namespace ImGui

Setting Up Contexts

mainCtx is a pointer to the main ImGuiContext instance. oskCtx is a pointer to the on-screen keyboard ImGuiContext instance.

The rest of the code assumes these contexts have been properly initialized beforehand. Something important I figured out is the backend must be initialized for all contexts. So all functions like ImGui_ImplOpenGL3_Init() must be called with each context active in order to work correctly. Same goes for the NewFrame and Render functions. Not sure if this applies to other backends, or just OpenGL.

Handling IO

// Called by the main application when a mouse button is pressed or released
void ApplicationGUI::OnMouseClick(int x, int y, int button, bool down) {
    ImGui::ScopedContextSwitch outer_scope(oskCtx);
    ImGuiIO &oskIO = ImGui::GetIO();
    oskIO.AddMouseButtonEvent(button, down);

    // Only send mouse click to main context if the keyboard didn't use it
    // Effectively z-layering IO events
    // I.e. user clicked in void around keyboard
    if(!oskIO.WantCaptureMouse) {
        ImGui::ScopedContextSwitch inner_scope(mainCtx);
        ImGui::GetIO().AddMouseButtonEvent(button, down);
    }
}

// Called by the main application when the mouse cursor position changes
void ApplicationGUI::OnMouseMove(int x, int y) {
    ImGui::ScopedContextSwitch outer_scope(oskCtx);
    ImGuiIO &oskIO = ImGui::GetIO();
    oskIO.AddMousePosEvent(x, y);

    // Same z-layering as OnMouseClick(), with keyboard having priority
    if(!oskIO.WantCaptureMouse) {
        ImGui::ScopedContextSwitch inner_scope(mainCtx);
        ImGui::GetIO().AddMousePosEvent(x, y);
    }
}

Render

// Called by main application when the GUI should render
void ApplicationGUI::RenderFrame(int vpW, int vpH) {
    // ---------------------- Main UI -----------------------
    ImGui::ScopedContextSwitch cur_scope(mainCtx);
    ImGui_ImplOpenGL3_NewFrame();
    ImGui::NewFrame();

    ImGuiIO &mainIO = ImGui::GetIO();
    // <calc IO DeltaTime, DisplaySize> if using custom backend

    if(ImGui::Begin("Main Window", nullptr, ImGuiWindowFlags_AlwaysAutoResize)) {
        static char text1[100] = "some text";
        ImGui::InputText("InputText1", text1, IM_ARRAYSIZE(text1));
        static char text2[100] = "more text";
        ImGui::InputText("InputText2", text2, IM_ARRAYSIZE(text2));
    }
    ImGui::End(); // Main Window

    ImGui::EndFrame();
    ImGui::Render();
    ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());

    // --------------- On-Screen Keyboard -----------------
    // IMPORTANT: Keyboard drawn AFTER main UI to ensure it's always rendered ON TOP
    ImGui::SetCurrentContext(oskCtx); // main ScopedContextSwitch still in scope so can set context directly
    ImGui_ImplOpenGL3_NewFrame();
    ImGui::NewFrame();

    ImGuiIO &oskIO = ImGui::GetIO();
    // <calc IO DeltaTime, DisplaySize> if using custom backend

    if(mainIO.WantTextInput) {
        ImGui::SetNextWindowPos(ImVec2(oskIO.DisplaySize.x * 0.5f, oskIO.DisplaySize.y * 0.5f), ImGuiCond_Always, ImVec2(0.5f,0.5f));
        ShowKeyboard(mainCtx);
    }

    ImGui::EndFrame();
    ImGui::Render();
    ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());
}

And here's an example keyboard with a letter key and backspace key:

bool ShowKeyboard(ImGuiContext *mainCtx) {
    bool keyboard_open = true;
    if(ImGui::Begin("Keyboard", &keyboard_open, ImGuiWindowFlags_AlwaysAutoResize | ImGuiWindowFlags_NoCollapse | ImGuiWindowFlags_NoResize)) {
        const ImVec2 keySize(40, 0);
        if(ImGui::Button("a", keySize)) {
            ImGui::ScopedContextSwitch cur_scope(mainCtx); // The main context must be active to push IO events
            ImGui::GetIO().AddInputCharacter('a');
        }
        ImGui::SameLine();
        if(ImGui::Button(ICON_FA_DELETE_LEFT, keySize)) {
            ImGui::ScopedContextSwitch cur_scope(mainCtx);
            ImGui::GetIO().AddKeyEvent(ImGuiKey_Backspace, true); // emulate backspace pressed
            ImGui::GetIO().AddKeyEvent(ImGuiKey_Backspace, false);
        }
    }
    ImGui::End(); // Keyboard window

    // Unfocus the InputText to avoid keyboard reopening due to io.WantTextInput still being true
    if(!keyboard_open) {
        ImGui::ScopedContextSwitch cur_scope(mainCtx);
        ImGui::GetIO().AddKeyEvent(ImGuiKey_Enter, true);
        ImGui::GetIO().AddKeyEvent(ImGuiKey_Enter, false);
    }
    return keyboard_open;
}

Hopefully others find this information useful.

[PathogenDavid]

Glad you got your multi-context solution working, and thanks for sharing your solution!

One thing you might add is disabling the mouse via AddMousePosEvent(-FLT_MAX, -FLT_MAX); for the mainCtx when oskCtx is owns it. I'm pretty sure that right now if you had a button bordering the edge of the OSK and hovered over it on the way to the OSK it would appear stuck in the hovered state.