google-deepmind
Print text on viewer window
Hello, I'm using the mujoco pybinding 3.0.1 and it's passive viewer to do some RL tasks. Could someone tell me how to print text on it? I tried mjr_overlay but I don't really understand the context and viewport part so it didn't work. I would appreciate it a lot if anyone could help me.
I'm not sure about the passive viewer, but it would be pretty easy to add this functionality to the regular mujoco.Renderer. Would that serve your needs?
I'm not sure about the passive viewer, but it would be pretty easy to add this functionality to the regular
mujoco.Renderer. Would that serve your needs?
Thanks for your reply ^^. Actually I can do that using the other viewer like mujoco-python-viewer, but some of my codes are deeply related to the native passive viewer, so I want to implement this functionality on it.
Another choice for me is to use mujoco-python-viewer instead. But when I use time.sleep in the simulation loop trying to make the animation slower, the mouse perturb in mujoco_python_viewer is also delayed while it's fine in the passive viewer. Like this:
# with mujoco_python_viewer
for _ in range(int(self.mocap.dt/model.opt.timestep)):
mujoco.mj_step(model, data)
viewer.render()
time.sleep(model.opt.timestep * 10)
# with passive_viewer
for _ in range(int(self.mocap.dt/model.opt.timestep)):
mujoco.mj_step(model, data)
viewer.sync()
time.sleep(model.opt.timestep * 10)
So if someone could help me with this, it would be a very nice solution too.
Sorry for the late reply, I'm currently off work and just noticed this.
IIRC it should be possible to add text labels as mjvGeoms into user_scn in the passive viewer.
Oh, I tried this, it's really nice! Through mujoco.mjtGeom.mjGEOM_LABEL, I can show text in the scene. Actually I was just going to print some text on the 2D UI in passive viewer, but this is also great. Thanks very much!!!
@hanyang9 would it be possible to post some code that worked for you? I encountered the same issues (beginner user here).
@hanyang9 would it be possible to post some code that worked for you? I encountered the same issues (beginner user here).
Sure.
def add_text(data, viewer, input):
# create an invisibale geom and add label on it
geom = viewer.user_scn.geoms[viewer.user_scn.ngeom]
mujoco.mjv_initGeom(
geom,
type=mujoco.mjtGeom.mjGEOM_LABEL,
size=np.array([0.2, 0.2, 0.2]), # label_size
pos=data.qpos[:3]+np.array([0.0, 0.0, 1.0]), # lebel position, here is 1 meter above the root joint
mat=np.eye(3).flatten(), # label orientation, here is no rotation
rgba=np.array([0, 0, 0, 0]) # invisible
)
geom.label = input # receive string input only
viewer.user_scn.ngeom += 1
and remember to clear the geoms in the scene every step by using:
viewer.user_scn.ngeom = 0
@hanyang9 thank you so much! I appreciate it! Do I need to modify (or assign) max geom limit?
@hanyang9 thank you so much! I appreciate it! Do I need to modify (or assign) max geom limit?
It depends on your need, the default viewer.user_scn.maxgeom=20000 which should be big enough. But the limitation of string length of label is 99, you may need to be careful with this if you want to show some long text or data in a label.
I really thank you so much for your great help! I implemented a minimal example with a viewer (basically it starts the mujoco GUI app) and I was able to see the text!
My goal was to create a video and be able to add a label or title to the video. As far as I read I should not use the viewer when recording. In the tutorial I found that mediapy is used to capture the video. I am not sure if that's the best way doing it but this is a minimal example which I am not sure how to modify to accommodate your code as here there is no viewer.
# Load the humanoid model
model = mujoco.MjModel.from_xml_path("./assets/humanoid.xml")
data = mujoco.MjData(model)
renderer = mujoco.Renderer(model)
# enable joint visualization option:
scene_option = mujoco.MjvOption()
scene_option.flags[mujoco.mjtVisFlag.mjVIS_JOINT] = True
duration = 3.8 # (seconds)
framerate = 60 # (Hz)
frames = []
step = 0
mujoco.mj_resetData(model, data)
while data.time < duration: # Run the simulation
if step < 200:
# Move the left hand toward the box using IK
joint_angles = calculate_inverse_kinematics(model, data, target_position, "hand_left_site")
for i, angle in zip(range(len(joint_angles)), joint_angles):
data.qpos[i] = angle
mujoco.mj_step(model, data) # Step the simulation forward
if len(frames) < data.time * framerate:
renderer.update_scene(data, scene_option=scene_option)
pixels = renderer.render()
frames.append(pixels)
step += 1
media.write_video('./frames/01.mp4', frames, fps=framerate)
This is what I wanted to do at first but always didn't have time to do this before, since you asked me, I tried a lot and now I finally solve it. I didn't expect that I can do this by myself before actually haha. Here is a simple example using mediapy.
frames = []
renderer = mujoco.Renderer(model, height, width)
for i in range(num_frames):
renderer.scene.ngeom = 0 # clear all the geom in the scene
mujoco.mj_forward(model, data)
# !!! update scene before adding your self-defined geom, if you update it after that, it will cover the geom you defined
renderer.update_scene(data)
# initialize the geom, here is a ball, if you want the label only, just change it to the mujoco.mjtGeom.mjGEOM_LABEL
geom = renderer.scene.geoms[renderer.scene.ngeom]
mujoco.mjv_initGeom(
geom,
type=mujoco.mjtGeom.mjGEOM_SPHERE,
size=np.array([0.1, 0.1, 0.1]), # label_size
pos=np.array([0.0, 0.0, 0]), # label position
mat=np.eye(3).flatten(), # label orientation
rgba=np.array([1, 0, 0, 1]) # red for the sphere
)
# add label
geom.label = "hello world"
# add geom into scene
renderer.scene.ngeom += 1
pixels = renderer.render()
frames.append(pixels)
# show the video
framerate = 30
media.show_video(frames, fps=framerate)
It turns out that mujoco.viewer is just a high-level well-packaged renderer, so actually renderer and viewer share the same logic.
@hanyang9 wow! At last a fully working example! That is so awesome! Thank you!
@saran-t Is it possible to display text in different colors rather than just white text?
mjvGeoms have an RGBA attribute. Does text ignore it? Maybe yes, I'm not sure.
@yuvaltassa I tried changing the RGBA array for mjvGeoms with mjGEOM_LABEL, but the text color didn’t change. I’m simply curious if there are any features or functions that support this.
Did you try using mjr_text or mjr_label? They both take RGB arguments, seems like one of them would do what you want?
cc @vikashplus
@yuvaltassa - IIUC the only thing we can do from the Python API is to add a geom with labels.
geom = renderer.scene.geoms[renderer.scene.ngeom]
mujoco.mjv_initGeom(
geom,
type=mujoco.mjtGeom.mjGEOM_SPHERE,
size=np.array([0.1, 0.1, 0.1]), # label_size
pos=np.array([0.0, 0.0, 0]), # label position
mat=np.eye(3).flatten(), # label orientation
rgba=np.array([1, 0, 0, 1]) # red for the sphere
)
# add label
geom.label = "hello world"
Is adding text directly supported via the python API?
All these functions are available via bindings, see e.g., here and here. True, they are not exposed in the mujoco.Renderer utility, but you can still use them. You're very welcome to add this functionality. For example dm_control exposed the text overlay functionality. You could add a similar thing to mujoco.Renderer.