Issue moving mujoco gameobject with transform.position in unity

[michaeljrose]

Hi,

I'm a student and I'm trying to use the MuJoCo Unity plug-in and HAPTIX to study human performance of referred robotic arm control. I am running MuJoCo 3.1.1 on Unity 2022.3.5f1

Currently I have an environment with a mujoco element in it (a cylinder with an mjbody, mj free joint, and mjgeom attached to it) and I want to be able to programmatically move it to different locations on command. Right now I have a C# script that I am trying to use to make it move using transform.position but when I do that the object flashes to the new desired location and then goes right back to where it was the frame before. I am guessing this comes from MuJoCo taking over the physics side of things and having an issue with Unity trying to do something physically impossible with the transform.position command? I was wondering if there is any way I could get around this issue or if I'll just have to destroy the old object and instantiate a new one where I want it to be to get what I want to happen?

Thanks! Michael

[Balint-H]

Sorry for missing this issue until now! Feel free to @ me in the future to bring a Unity plugin related question to my attention.

MjBody position is fully determined by the mujoco simulation, and changes in the Unity scene won't be persistent for the next frame (barring a MjScene reconstruction, which is one of your options but may have certain side effects).

If you just want to teleport the object to a new position you can do it the same way it can be done in regular MuJoCo, by editing the qpos array. Here's an example on how to do that:

https://github.com/Balint-H/modular-agents/blob/927a4ead43c8b8e4db902657f37550db3b7d45e9/mj-unity-extensions/Runtime/MjState.cs#L143

If you want full continuous control of an MjBody so the state is only determined by you in Unity, you can use an MjMocapBody, or connecting the MjBody to an MjMocapBody through a weld constraint if you want to allow some deviation from your prescribed position.

[michaeljrose]

Hi Balint,

Thanks for the response, being able to teleport the object would be great. However, I am confused as to how to implement the example you linked. The TeleportMjRoot function looks straightforward enough to use, but I'm not sure how to get to that point within the scripts I currently have -- would I put the MjState.cs in my local .../unity/Runtime level and then call it from my project's scripts or is there some other way?

Thanks, Michael

[Balint-H]

Hello, apologies for missing your message for so long... the approach you describe is indeed how its meant to be used. Of course if you want you can copy the relevant parts of MjState to your script as well. Let me know if you since moved on from this problem.