drake
drake copied to clipboard
Remove X server requirement from OpenGL and VTK renderers
Is your feature request related to a problem? Please describe. According to @jwnimmer-tri, there's no reason for an X server to be a strict requirement for these renderers. It makes it harder to deploy to headless environments for containerized environments.
Describe the solution you'd like RenderEngineGL could use EGL to create it's OpenGL context. VTK can automatically use EGL if built with that option.
Describe alternatives you've considered
I considered using xvfb
but that does non-HW-accelerated rendering and would be a bottleneck.
FYI on a related topic...
As of Bazel 7.0, to access a real Xorg server during Bazel tests downstream projects must add this line in their .bazelrc
:
build --sandbox_add_mount_pair=/tmp/.X11-unix
Otherwise, you'll see an failure such as DRAKE_THROW_UNLESS(display != nullptr);
around internal_opengl_context.cc:85
.
This is due to the following change in Bazel 7: --incompatible_sandbox_hermetic_tmp
is enabled by default.
I tried some prototyping today...
I used this WIP branch that switches VTK rendering to use EGL instead of GLX. I built a wheel file, installed it on Deepnote, and did apt install libegl-mesa0
instead of install_xvfb
. I ran rendering_multibody_plant.ipynb
and it worked fine. Thus, EGL seems like a viable option for CPU-based rendering without an X server.
As I understand it, there are ways to teach EGL to use the GPU, but I haven't looked into that specifically yet.
When using EGL, the camera option to show_window = True
seemingly has no effect, at least not in my WIP branch. Possibly some device configuration is necessary, or maybe I need to be using Wayland instead of Xorg. Also, most of the VTK render engine tests pass, but two cases fail (for me).
Assuming that EGL can't be made strictly better than GLX, the next steps would be to see if we can compile both GLX and EGL support into Drake's VTK, and offer a RenderEngineVtkParams
option for which one to use.
For our Kitware helpers -- let's say the first milestone on this issue is just RenderEngineVtk. We'll leave the RenderEngineGl for a second milestone.
Actually, I take that back. Let's use #21700 for the VTK option, and keep this ticket for the RenderEngineGl changes.