How to make screen capture and video recording working in backend examples of ImGui

[nconsulting]

Thank you for fixing issue #69. I managed to get the app_minimal example to work. I'm now trying to get the screen capture and video recording to work in my own ImGui code. I have everything else from the test engine working in my own code. My ImGui code is based on the code in examples\example_glfw_opengl3.

In the app_minimal example an ImGuiApp instance is created in main. This is not the case in the ImGui examples, further more this ImGuiApp instance is also used in the ImGuiApp_ScreenCaptureFunc function. I have tried a couple of things, but I'm unable to get it to work and I haven't been able to find any examples that don't use the ImGuiApp instance. Can you point me to an example that has the same code structure as the examples in ImGui or should I switch to a similar setup as the app_minimal example or can you give me some pointers? Thank you!

[ocornut]

Capturing the framebuffer is dependent on your rendering stack/backend. imgui_app is a small/local helper we use. ImGuiApp_ScreenCaptureFunc() will essentially call ImGuiApp_ImplGlfwGL3_CaptureFramebuffer() which does:

#ifdef IMGUI_HAS_VIEWPORT
    if (ImGui_ImplGlfw_ViewportData* vd = (ImGui_ImplGlfw_ViewportData*)viewport->PlatformUserData)
        if (vd->Window)
            glfwMakeContextCurrent(vd->Window);
#endif
  return ImGuiApp_ImplGL_CaptureFramebuffer(app, viewport, x, y, w, h, pixels, user_data);

static bool ImGuiApp_ImplGL_CaptureFramebuffer(ImGuiApp* app, ImGuiViewport* viewport, int x, int y, int w, int h, unsigned int* pixels, void* user_data)
{
    IM_UNUSED(app);
    IM_UNUSED(user_data);
    IM_UNUSED(viewport); // Expecting calling code to have set the right GL context

#ifdef __linux__
    // FIXME: Odd timing issue is observed on linux (Plasma/X11 specifically), which causes outdated frames to be captured, unless we give compositor some time to update screen.
    // glFlush() didn't seem enough. Will probably need to revisit that.
    usleep(1000);   // 1ms
#endif

    int y2 = (int)ImGui::GetIO().DisplaySize.y - (y + h);
    glPixelStorei(GL_PACK_ALIGNMENT, 1);
    glReadPixels(x, y2, w, h, GL_RGBA, GL_UNSIGNED_BYTE, pixels);

    // Flip vertically
    size_t comp = 4;
    size_t stride = (size_t)w * comp;
    unsigned char* line_tmp = new unsigned char[stride];
    unsigned char* line_a = (unsigned char*)pixels;
    unsigned char* line_b = (unsigned char*)pixels + (stride * ((size_t)h - 1));
    while (line_a < line_b)
    {
        memcpy(line_tmp, line_a, stride);
        memcpy(line_a, line_b, stride);
        memcpy(line_b, line_tmp, stride);
        line_a += stride;
        line_b -= stride;
    }
    delete[] line_tmp;
    return true;
}
#endif

So you should need to copy those into your app.

[ocornut]

In theory, I could decide that standard imgui backend could provide this helper function as part of the backend, but it would mean dragging code in imgui backends that is solely necessary for users of the Test Engine and capturing system. It seems a little odd to do that.

[nconsulting]

Thank you for your quick reply. I still don't get it though. My C++ knowledge is too limited for this. I tried copying the code you provided in my app. But that basically presents the same problem I had before. I don't have an ImGuiApp app, nor a ImGuiViewport viewport object which are needed in ImGuiApp_ImplGL_CaptureFramebuffer. In order to get those I need to copy the shared/imgui_app.h code in my app (and probably a couple of more files), or so it seems to me. Currently my code is setup like in https://github.com/ocornut/imgui/blob/master/examples/example_glfw_opengl3/main.cpp .

[ocornut]

The function which is assigned to test_io.ScreenCaptureFunc is the following:

    test_io.ScreenCaptureFunc = ImGuiApp_ScreenCaptureFunc;
    test_io.ScreenCaptureUserData = (void*)app;

bool ImGuiApp_ScreenCaptureFunc(ImGuiID viewport_id, int x, int y, int w, int h, unsigned int* pixels, void* user_data)
{
    ImGuiApp* app = (ImGuiApp*)user_data;
    if (app->CaptureFramebuffer == NULL)
        return false;
#ifdef IMGUI_HAS_VIEWPORT
    ImGuiViewport* viewport = ImGui::FindViewportByID(viewport_id);
#else
    ImGuiViewport* viewport = ImGui::GetMainViewport();
    IM_UNUSED(viewport_id);
#endif
    IM_ASSERT(viewport != NULL);
    return app->CaptureFramebuffer(app, viewport, x, y, w, h, pixels, NULL);
}

You should probably set a breakpoint in it to step into the code so you'll understand the hoops it is doing, then you can flatten it into your own function (you don't need the hoops in your own code).

[ocornut]

Are you using multi-viewports? I just realized of an issue since the code access backend's internal structure to get the GLFW window.

[ocornut]

I have pushed two commits acbcde0 and 2d2707a to make the code in imgui_app easier to reuse.

Here's is the code simplified and flattened for GLFW+OpenGL:

    ImGuiTestEngineIO& test_io = ImGuiTestEngine_GetIO(engine);
    test_io.ScreenCaptureFunc = ExampleGlfwGL3_ScreenCaptureFunc;
    test_io.ScreenCaptureUserData = nullptr;

static bool ExampleGlfwGL3_ScreenCaptureGL(ImGuiViewport* viewport, int x, int y, int w, int h, unsigned int* pixels)
{
#ifdef __linux__
    // FIXME: Odd timing issue is observed on linux (Plasma/X11 specifically), which causes outdated frames to be captured, unless we give compositor some time to update screen.
    // glFlush() didn't seem enough. Will probably need to revisit that.
    usleep(1000);   // 1ms
#endif

    int y2 = (int)viewport->Size.y - (y + h);
    glPixelStorei(GL_PACK_ALIGNMENT, 1);
    glReadPixels(x, y2, w, h, GL_RGBA, GL_UNSIGNED_BYTE, pixels);

    // Flip vertically
    size_t comp = 4;
    size_t stride = (size_t)w * comp;
    unsigned char* line_tmp = new unsigned char[stride];
    unsigned char* line_a = (unsigned char*)pixels;
    unsigned char* line_b = (unsigned char*)pixels + (stride * ((size_t)h - 1));
    while (line_a < line_b)
    {
        memcpy(line_tmp, line_a, stride);
        memcpy(line_a, line_b, stride);
        memcpy(line_b, line_tmp, stride);
        line_a += stride;
        line_b -= stride;
    }
    delete[] line_tmp;
    return true;
}

bool ExampleGlfwGL3_ScreenCaptureFunc(ImGuiID viewport_id, int x, int y, int w, int h, unsigned int* pixels, void* user_data)
{
    IM_UNUSED(user_data);

    // Make context current when using multiple viewports
#ifdef IMGUI_HAS_VIEWPORT
    ImGuiViewport* viewport = ImGui::FindViewportByID(viewport_id);
    if (GLFWwindow* window = (GLFWwindow*)viewport->PlatformHandle)
        glfwMakeContextCurrent(window);
#else
    ImGuiViewport* viewport = ImGui::GetMainViewport();
#endif

    // OpenGL capture
    return ExampleGlfwGL3_ScreenCaptureGL(viewport, x, y, w, h, pixels);
}

But I noticed it seems like there is a bug capturing in multi-viewport mode right now (investigating).

[nconsulting]

Yes, I'm using multi-viewports. And I also added code for a dockspace.

[ocornut]

ImGuiCaptureFlags_StitchAll seems to have a bug with multi-viewports right now (see #33), but other than that it should work.

[nconsulting]

Thank you for making all these adjustments and for your code examples. I copied ExampleGlfwGL3_ScreenCaptureGL and ExampleGlfwGL3_ScreenCaptureFunc to my main.cpp and also changed test_io.ScreenCaptureFunc and test_io.ScreenCaptureUserData. I also removed ImGuiCaptureFlags_StitchAll from the capture_screenshot test, just in case.

I retested, but unfortunately, I still get an error: Error imgui_te_context.cpp:708 'window != nullptr'

I also set breakpoints in ExampleGlfwGL3_ScreenCaptureGL and ExampleGlfwGL3_ScreenCaptureFunc, but they don't get hit. It seems like both functions are not called.