mujoco Issue with Plugin and Sensor Size Constraint

Intro

Hi!

I am a researcher at Institute for Artificial Intelligence, University of Bremen, I use MuJoCo for my research.

My setup

MuJoCo: 3.2.7 API: C OS: Ubuntu 20.04 / 24.04

What's happening? What did you expect?

I'm trying to implement a custom plugin for my use case, using the sensor plugin as a starting point. While working with touch_grid.xml, I noticed something unexpected:

In the XML, there is only one sensor attached to the site named "touch". However, during debugging, I observed the following behavior with the lambda function plugin.nsensordata:

It's called twice.
In the first call, the values are always:
- m->sensor_objid[sensor_id] = 0
- m->sensor_objtype[sensor_id] = mjOBJ_UNKNOWN
In the second call, the values are correct:
- m->sensor_objid[sensor_id] and m->sensor_objtype[sensor_id] have the expected values.

Upon further investigation, I found that:

The first call determines sensor_size and the second call determines m->nsensordata to satisfy the constraint mentioned in engine_io.c.

This behavior creates a challenge for my use case because:

My sensor size depends on m->sensor_objid[sensor_id] and m->sensor_objtype[sensor_id].
This causes the constraint to be violated.

Questions:

How can I address this issue and ensure my sensor size is calculated correctly without violating the constraint?
I also noticed that the constructor for the sensor is called twice. Is this redundant, or is it expected behavior?

Any insights or suggestions would be greatly appreciated. Thank you!

Steps for reproduction

Open file touch_grid.cc.
Add from line 506 the following codes:

const int obj_id = m->sensor_objid[sensor_id];
const char *obj_name = mj_id2name(m, mjtObj::mjOBJ_SITE, obj_id);
const int obj_type = m->sensor_objtype[sensor_id];
printf("Object id: %d\n", obj_id);
printf("Object name: %s\n", obj_name);
if (obj_type == mjOBJ_SITE)
{
  printf("Object is of type mjOBJ_SITE\n");
}
else if (obj_type == mjOBJ_UNKNOWN)
{
  printf("Object is of type mjOBJ_UNKNOWN\n");
}
else
{
  printf("Object is of type %d\n", obj_type);
}

Build it and simulate touch_grid.xml or the below XML (with a dummy site).
The output would be:

Object id: 0
Object name: dummy
Object is of type mjOBJ_UNKNOWN
Object id: 1
Object name: touch
Object is of type mjOBJ_SITE

Minimal model for reproduction

<mujoco model="touchtest">
  <compiler autolimits="true"/>
  <extension>
    <plugin plugin="mujoco.sensor.touch_grid"/>
  </extension>
  <asset>
    <texture name="grid" type="2d" builtin="checker" rgb1=".1 .2 .3" rgb2=".2 .3 .4" width="300" height="300" mark="edge" markrgb=".2 .3 .4"/>
    <material name="grid" texture="grid" texrepeat="3 1" texuniform="true"/>
    <hfield file="a.png" size="0.2 0.2 0.02 0.015"/>
  </asset>
  <visual>
      <headlight ambient=".7 .7 .7" diffuse=".2 .2 .2" specular="0.1 0.1 0.1"/>
      <map znear="0.01"/>
      <scale contactwidth=".02" contactheight=".5"/>
  </visual>
  <default>
    <geom friction="0.4" solimp="0 0.95 0.02"/>
  </default>
  <statistic center="0 0 1" extent="1" meansize=".1"/>
  <worldbody>
    <light pos="1 0 .3" dir="-1 0 -.3"/>
    <light pos="-1 0 .3" dir="1 0 -.3"/>
    <geom name="floor" pos="0 0 -0.01" type="plane" size="1 1 .01"/>
    <geom name="a" type="hfield" hfield="a" rgba=".5 .5 .7 1"/>
    <body name="ball" pos="0 0 1">
      <joint name="x" type="slide" axis="1 0 0" damping="1"/>
      <joint name="y" type="slide" axis="0 1 0" damping="1"/>
      <joint name="z" type="slide" axis="0 0 1"/>
      <joint name="rx" axis="1 0 0" springdamper="0.2 1"/>
      <joint name="ry" axis="0 1 0" springdamper="0.2 1"/>
      <geom type="ellipsoid" size=".3 .5 .1" mass="0.1" rgba=".5 .5 .5 .3"/>
      <site name="dummy"/>
      <site name="touch" pos="0 0 .1"/>
    </body>
  </worldbody>
  <sensor>
    <plugin name="touch" plugin="mujoco.sensor.touch_grid" objtype="site" objname="touch">
      <config key="size" value="7 7"/>
      <config key="fov" value="45 45"/>
      <config key="gamma" value="0"/>
      <config key="nchannel" value="3"/>
    </plugin>
  </sensor>
</mujoco>

Code required for reproduction

No response

Confirmations

[X] I searched the latest documentation thoroughly before posting.
[X] I searched previous Issues and Discussions, I am certain this has not been raised before.

Dec 04 '24 09:12 HoangGiang93

Could you please elaborate on why you need your sensor size to depend on m->sensor_objid[sensor_id] and m->sensor_objtype[sensor_id]? This strikes me as a strange requirement...

Jan 03 '25 17:01 saran-t

I was working on developing a connector that links MuJoCo data to a structure for subsequent transmission over TCP. I managed to accomplish this without using sensors. Despite this, I believe my observations remain partially valid, particularly since the initialization function always called twice.

Jan 03 '25 17:01 HoangGiang93

@HoangGiang93 do you still think this needs to be addressed ?

Jul 21 '25 12:07 yuvaltassa

Well it doesn't bother me anymore, but it might bother someone else in the future.

Jul 22 '25 06:07 HoangGiang93