Severe Lag When Dragging Chrome/Warp-Terminal Windows with Mujoco Simulate Viewer Open, Possibly Related to Rendering or V-Sync

[tqolf]

Intro

Severe Lag and high CPU load

My setup

MuJoCo 2.3.3, Python amd64, Linux/Ubuntu 22.04

What's happening? What did you expect?

1. When running the simulation with no other actions, CPU usage stays around 2-10%, and dragging the built-in Ubuntu terminal is still very smooth.
2. With the simulation running under default settings, dragging Chrome or Warp-Terminal results in CPU usage spiking to 80-100%, with severe lag. perf shows sched_yield accounting for over 80% of the CPU time.
3. When running the simulation with Vertical Sync disabled, CPU usage reaches 100%, but the lag when dragging Chrome/Warp-Terminal improves significantly, though there is still noticeable lag at the beginning of the drag.

Steps for reproduction

See

Minimal model for reproduction

Maybe unrelated to the model.

Code required for reproduction

Maybe unrelated.

Confirmations

• [X] I searched the latest documentation thoroughly before posting.
• [X] I searched previous Issues and Discussions, I am certain this has not been raised before.

[yuvaltassa]

You are using a very old version. Does this also happen with latest MuJoCo?

[tqolf]

Yeah, there is a plan to upgrade, but we are still working on migration. Currently, we want to first understand what is causing this phenomenon.

[tqolf]

You are using a very old version. Does this also happen with latest MuJoCo?

I tried to upgrade, but the new code is very slow, elasped_time > 0.008

import mujoco
import mujoco.viewer

m = mujoco.MjModel.from_xml_path(xx)
d = mujoco.MjData(m)

viewer = viewer.launch_passive(m, d)
viewer.sync()

while viewer.is_running():
       last_time = time.time()
      # ...
      mujoco.mj_step(m, d)
      viewer.sync()

     elasped_time = time.time() - last_time
     if elasped_time < 0.001:
           await asyncio.sleep(0.001 - elasped_time)

[tqolf]

You are using a very old version. Does this also happen with latest MuJoCo?

And I have tried the latest MuJoCo, Severe Lag still exists when I draging Chrome/Warp-Terminal.

[yuvaltassa]

Can you please post a video of what you're seeing?

[tqolf]

Screencast from 2024年10月14日 21时30分15秒.webm

Can you please post a video of what you're seeing?

[tqolf]

You are using a very old version. Does this also happen with latest MuJoCo?

I tried to upgrade, but the new code is very slow, elasped_time > 0.008

import mujoco
import mujoco.viewer

m = mujoco.MjModel.from_xml_path(xx)
d = mujoco.MjData(m)

viewer = viewer.launch_passive(m, d)
viewer.sync()

while viewer.is_running():
       last_time = time.time()
      # ...
      mujoco.mj_step(m, d)
      viewer.sync()

     elasped_time = time.time() - last_time
     if elasped_time < 0.001:
           await asyncio.sleep(0.001 - elasped_time)

@yuvaltassa And How about this? When I use the old version, it's no need to call view.sync(). Also, the C++ version didn't require it and ran smoothly. Does python sync API synchronize all the data for rendering? Are there any alternative optimization methods?

[tqolf]

Screencast.from.2024.10.14.21.30.15.webm

Can you please post a video of what you're seeing?

And I use perf to analyze performance, sched_yield accounting for over 80% of the CPU time.

[yuvaltassa]

@saran-t might be able to help

[saran-t]

What GPU are you using?

[tqolf]

What GPU are you using?

CPU: 13th Gen Intel(R) Core(TM) i7-13700K GPU: NVIDIA GeForce RTX 4060

And I think GPU is not releated to this problem, 'cause sched_yield too frequently and GPU usage is very slow

[tqolf]

@saran-t @yuvaltassa Just wanted to check if there are any updates on the issue. It’s been a little while, and I wanted to see if it’s still on the radar.

the latest version, and my application like this

import mujoco
import mujoco.viewer

m = mujoco.MjModel.from_xml_path(xx)
d = mujoco.MjData(m)

viewer = viewer.launch_passive(m, d)
viewer.sync()

while viewer.is_running():
       last_time = time.time()
      # ...
      mujoco.mj_step(m, d)
      viewer.sync()

     elasped_time = time.time() - last_time
     if elasped_time < 0.001:
           await asyncio.sleep(0.001 - elasped_time)

Perf shows memmove is consuming 91% CPU. Simulate::Sync has many copies.

_launch_internal

_launch_internal:
 ....
  if model and not run_physics_thread:
    user_scn = mujoco.MjvScene(model, _Simulate.MAX_GEOM)
  else:
    user_scn = None
  simulate = _Simulate(
      cam, opt, pert, user_scn, run_physics_thread, key_callback
  )

simulate.cc

will it cause high CPU usage ?

[saran-t]

Do you know where the memmove is coming from?

[tqolf]

I'm sure it comes from Simulate::Sync, but I have no idea about which branch, maybe need debug version to check it.

[tqolf]

@saran-t mjv_updateSceneState should be responsible. How can I achieve incremental updates? I have provide d.ctrl = map{id -> data}, which id get via mj_name2id.

[saran-t]

Hang on, you're targeting 1000 updates per second (if elasped_time < 0.001), that's almost definitely where the problem is.

Try targeting something like 60-100 updates per second instead. If you need to update your physics more frequently then do that in an inner loop (i.e. call mj_step multiple times per viewer update).

[tqolf]

Hang on, you're targeting 1000 updates per second (if elasped_time < 0.001), that's almost definitely where the problem is.

Try targeting something like 60-100 updates per second instead. If you need to update your physics more frequently then do that in an inner loop (i.e. call mj_step multiple times per viewer update).

Got it, I'll try later.

[yuvaltassa]

Assuming this is fixed since your attempt at 1000Hz render FPS is clearly untenable.

@saran-t perhaps we should introduce a hard cap in the code to avoid this in future?