Access to command buffer to modify ImGui DrawList through ImDrawList::AddCallback

[dafedidi]

Version/Branch of Dear ImGui:

Version: 1.88 Branch: docking

Back-end/Renderer/Compiler/OS

Back-ends: imgui_impl_Vulkan.cpp Compiler: MSVC_2019 Operating System: Windows

My Issue/Question:

How do you access the currently bound command buffer?

Currently, there is no way to access a command buffer during a ImDrawList::AddCallback. This is because the moment the draw call is actually made (ImGui_ImplVulkan_RenderDrawData) is called much later than when ImGui start rendering the whole frame. At least, that the reason I believe for that.

However, it makes it impossible for the user to modify the currently bound pipeline/shader to let it affect the next ImGui rendering commands if the user wants to. Such example could be activating and removing blending on the currently bound pipeline ( the default is always active) or rendering with custom shaders ImGui's text, headers, etc...

For example, I have a simple image rendered before and I want to display it in a viewport. This is the end result.

Here, the text has been rendered with a pipeline that had blending active and so the final texture has an alpha component that varies between 0 and 1. However, when rendered, since the default pipeline has blending enabled and we can't access the command buffer, it's impossible to change that there.

As proof, here is the rendered texture in renderdoc.

At the moment, my only way to patch that is to run one more pipeline BEFORE IMGUI to remove the alpha channel. Using OpenGL would be a different story because it works as a state machine. That is not the case with Vulkan and the way it's currently made, there is, as far as I know, no other possibilities.

If you DO know a way to bypass that limitation, please let me know.

Here is the culprit code, probably the same for most modern backend (meaning metal, DX12 and Vulkan).

for (int n = 0; n < draw_data->CmdListsCount; n++)
    {
        const ImDrawList* cmd_list = draw_data->CmdLists[n];
        for (int cmd_i = 0; cmd_i < cmd_list->CmdBuffer.Size; cmd_i++)
        {
            const ImDrawCmd* pcmd = &cmd_list->CmdBuffer[cmd_i];
            if (pcmd->UserCallback != nullptr)
            {
                if (pcmd->UserCallback == ImDrawCallback_ResetRenderState)
                    ImGui_ImplVulkan_SetupRenderState(draw_data, pipeline, command_buffer, rb, fb_width, fb_height);
                else
                    pcmd->UserCallback(cmd_list, pcmd); // <-- No access to command buffer here
            }
            else // <----- Subsquent commands here will not be affected by a change of pipeline because the command buffer can't record the change
            {
                // [...] // <--- OMITTED CODE for clarity

               // <-------- Following command use the same command_buffer that the user can't record commands into
                vkCmdBindDescriptorSets(command_buffer, VK_PIPELINE_BIND_POINT_GRAPHICS, bd->PipelineLayout, 0, 1, desc_set, 0, nullptr); 

                vkCmdDrawIndexed(command_buffer, pcmd->ElemCount, 1, pcmd->IdxOffset + global_idx_offset, pcmd->VtxOffset + global_vtx_offset, 0);
            }
        }

Standalone, minimal, complete and verifiable example:

ImGui::Begin("Viewport", (bool*)0, ImGuiWindowFlags_NoDecoration | ImGuiWindowFlags_NoBackground);

ImDrawList* draw_list = ImGui::GetWindowDrawList();
draw_list->AddCallback(bind_pipeline_cmd, nullptr); // <-- no usercallback data in example, put what you need to test

ImVec2 viewport_available_region_size = ImGui::GetContentRegionAvail();
ImGui::Image(YOUR_HANDLE, viewport_available_region_size); // <-- Replace YOUR_HANDLE with a valid DescriptorSet

draw_list->AddCallback(ImDrawCallback_ResetRenderState, nullptr);
ImGui::End();

[PathogenDavid]

ImDrawList::AddCallback is indeed quite a bit less useful with the modern rendering APIs. I don't think there's a practical way to manipulate things like the blend state with them.

Even if we exposed the command buffer it'd be impractical to change the blend state independently since you'd also need to clone the entire pipeline and have the information needed to do so. (At least that's true for D3D12, I'm less familiar with Vulkan but I believe this applies there as well.)

I think the proper solution here would be to just clear the alpha channel of the image yourself after you're done rendering it like you mentioned.

[dafedidi]

Even if we exposed the command buffer it'd be impractical to change the blend state independently since you'd also need to clone the entire pipeline and have the information needed to do so. (At least that's true for D3D12, I'm less familiar with Vulkan but I believe this applies there as well.)

It's the same for Vulkan, since it's the way these API works. But it's possible, and it's fine, because the HLSL and GLSL used is available in both ImGui_Impl_XXX files and we can re-create the whole pipeline object and modify it for our needs. However, it seems to me that ImDrawList::AddCallback is not fullfulling that role for these APIs.

The way I see it would be to simply discard the ImGui::XXX command and roll our own command which is quite problematic. For example, instead of ImGui::Image, I could draw the image in that space through content area. But even that solution is not good because we don't know the order of operations of commands and we can't schedule it. We can't introduce a barrier or any such thing.

I believe that low level access like that could be added to the ImDrawCmd however, the ImGui_ImplXXX_RenderDrawData is using const during the rendering loop and the ImDrawCmd can't be modify during the loop to introduce the command buffer. It would be the best way to access it as a void* data. User code can and will be able to transform it into the right structure.

[PathogenDavid]

Is there a reason why clearing the alpha channel doesn't work for you?

[ocornut]

I think it would be reasonable to expose a backend-dependant structure including a pointer to dx12/vulkan command buffer. Question is where to add this void* without breaking api, as changing the draw callback signature would break existing backend (because function pointers types can’t specify defaut parameters).

[dafedidi]

Is there a reason why clearing the alpha channel doesn't work for you?

For this specific case, not really. Apart from the fact that codes that don't belong at one place, will be there with a big ol' comment. However, I believe the issue is deeper than that, because it prevents any usage of AddCallback in my codebase.

Question is where to add this void* without breaking api, as changing the draw callback signature would break existing backend (because function pointers types can’t specify defaut parameters).

Not sure to fully understand. You mean changing the structure that is received in parameter ( ImDrawCmd ) or changing the function signature (ImDrawCallback) ? I believe that changing the ImDrawCmd should not break existing API user by adding a field to it, but maybe I'm missing something.