Screen capture of multiple meshes

[Bachery]

Tutorial 607_ScreenCapture use viewer.core.draw_buffer() to capture the image of one mesh in the data_list of the viewer. If I draw multiple objects like tutorial 107_MultipleMeshes, how can I capture an image of the whole scene?

[jdumas]

I don't think it's possible with the current setup. Probably ViewerCore::draw_buffer() should take as input a lambda function for the explicit draw callback to be made (so that you can draw 1 mesh or several).

[xarthurx]

Instead of the viewer.data, is it possible to take a screen capture of the whole opengl buffer?

[jdumas]

Well, it's possible if you call yourself glReadPixels() on the framebuffer, but then you're gonna see the UI as well, and the resolution will be the resolution of the window.

[xarthurx]

I'm fine with that, because I use nanovg to draw something onto the buffer, but are not stored in the viewer.data.

Any suggestions where I should look at for an implementation? I'm not very familiar with the OpenGL stuff. Thank you in advance.

[jdumas]

You can look at how ViewerCore::draw_buffer() is implemented and try to adapt it to your needs.

[jhwang7628]

This will be really helpful to have..

[w-m]

I implemented a way to capture the whole scene as a ViewerPlugin that just does a glReadPixels in post_draw. If this capture plugin is the first plugin in the plugin list, then any UI that is rendered as a plugin afterwards is not visible in the captured output (e.g. an ImGUI menu).

Find some sample code below that also adds a button to start the capture. It'll pop up a file chooser to select a filename prefix and it will then capture to filenname%idx.png, with the idx just counting up from 0 until the stop capture button is clicked.

Writing the PNGs is done in a detached thread to keep the rendering responsive. Memory is allocated and a thread is created for every frame. In a situation where the PNG writing were slower than the rendering, then the memory usage would grow during the capture.

#include <igl/opengl/glfw/imgui/ImGuiMenu.h>
#include <igl/opengl/glfw/Viewer.h>
#include <igl/png/writePNG.h>

#include <memory>


class CapturePlugin : public igl::opengl::glfw::ViewerPlugin {
public:
  CapturePlugin() { plugin_name = "capture"; }

  bool post_draw() override {
    if (!capturing) {
      return false;
    }

    const int width  = viewer->core.viewport(2);
    const int height = viewer->core.viewport(3);

    std::unique_ptr<GLubyte[]> pixels(new GLubyte[width * height * 4]);

    glReadPixels(0, 0, width, height, GL_RGBA, GL_UNSIGNED_BYTE, pixels.get());

    std::string path = pathPrefix + std::to_string(captureIdx++) + ".png";
    std::thread{ writePNG, path, std::move(pixels), width, height }.detach();

    return false;
  }

  void startCapture(std::string capturePath) {
    pathPrefix = capturePath;
    captureIdx = 0;
    capturing  = true;
  }

  void stopCapture() { capturing = false; }

  bool isCapturing() const { return capturing; }

private:
  static void writePNG(std::string path, std::unique_ptr<GLubyte[]> pixels, int width, int height) {
    igl::stbi_write_png(path.c_str(), width, height, 4, pixels.get() + width * (height - 1) * 4, -width * 4);
  };

  std::string pathPrefix;
  int64_t     captureIdx;
  bool        capturing = false;
};

int main(int argc, char *argv[]) {
  igl::opengl::glfw::Viewer viewer;

  CapturePlugin capture;
  viewer.plugins.push_back(&capture);

  igl::opengl::glfw::imgui::ImGuiMenu menu;

  menu.callback_draw_viewer_menu = [&]() {
    if (!capture.isCapturing()) {
      if (ImGui::Button("Start capture", ImVec2(-1, 0))) {
        std::string capturePath = igl::file_dialog_save();
        if (!capturePath.empty()) {
          capture.startCapture(capturePath);
        }
      }
    } else {
      if (ImGui::Button("Stop capture", ImVec2(-1, 0))) {
        capture.stopCapture();
      }
    }
  };

  viewer.launch();
}

The code works for me, but I'm not sure if there are any gotchas. Maybe you could have a look @jdumas. I'll be happy to contribute this as example code if you think it's a sane solution and that it would be useful to others.

[jdumas]

One of the advantage of draw_buffer() and using an offscreen texture buffer is that you can crank up the resolution and do some downsampling afterwards to get a cheap antialiasing effect. A modification we should make to this draw_buffer() is to be able to take a lambda function to specify exactly what to draw to take the screenshot, maybe with some overload to be able to easily render 1 or several meshes.

[w-m]

I believe there are two different use cases at play here.

My intention was to simply capture frames continuously from the window, e.g. while animating a mesh. Like with a screen recorder, but synced to the frames the Viewer produces (and less glitchy). So the code above works perfectly fine for me, I get exactly what I see. I'm fine with the resolution (I can resize the window) and the quality.

draw_buffer couldn't really be used for this use case in its current form - the setup and teardown of buffers and textures it does is too expensive to do at every frame. It's more of a high-quality single-shot capture.

I guess both use cases couldn't be easily covered by the same code.

[jdumas]

I see, makes sense. Then yes your approach looks fine to me!

[jhwang7628]

Thanks. The provided solution works perfectly for me (except that I need to make viewer.core -> viewer.core() to accommodate the recent change in multi-core viewer).