How to add tracking frames to Rigidbody

[zoctipus]

For example if I have a knife which has blade, and handle. I am interested in tracking the tip of the blade and the center of the handle using Transform

I am not able to use FrameTransformer because it requires and source prim_path and target_frames's prim_path defined, and both prim at prim_path needs to be rigidbodies.

In the scene Config, I can't assign it as RigidBodyCfg. Because in order to use FrameTransformer, I need to supply both source prim_path and target_frames's prim_path. But RigidBodyCfg only allow 1 Rigidbody be found under the usd file.

I can't assign it as Articulation. Even if I assign Rigidbody separately to blade and handle, and assign fixed joint, and define articulation root. It seems like Physx does not like articulation to be inmoveable.

So I wander how can I track some specific point on the rigid body, thank you.

[Mayankm96]

The knife is a rigid body, right? In that case, you can just pass its prim path as both source and target prim paths. Then you can set the "offset" for the knife's blade and handle center.

[zoctipus]

Yes! That is what I did, however the following implementation in frame_transformer.py removes the target frame if the body name is the same as the source frame. refers line 234

        # -- source frame
        self._source_frame_body_name = self.cfg.prim_path.split("/")[-1]
        source_frame_index = first_env_body_names.index(self._source_frame_body_name)
        # -- target frames
        self._target_frame_body_names = first_env_body_names[:]
        self._target_frame_body_names.remove(self._source_frame_body_name)

Then later on an error would occur at line 270

self._target_frame_offset_pos = torch.stack(target_frame_offset_pos).repeat(self._num_envs, 1)

due to target_frame_offset_pos is an empty list RuntimeError: stack expects a non-empty TensorList because self._target_frame_body_names is empty list

[Mayankm96]

Good point. This is a known issue. @jsmith-bdai do you think this is something we can easily fix? Or you have some insights on how we can go about it?

[jsmith-bdai]

Unfortunately I haven't dug into this yet as I've been busy with other tasks. The FrameTransformer code got quite convoluted as we added more functionality, so the plan was to overhaul it at some point rather than keep hacking at it.

I will take another stab at it this week and get back to you @zoctipus

[zoctipus]

Thank you so much for your guys hard work. And please take your time!