How can I get the bounding box of the Actor and Articulation

[DevinJoeUU]

I use the function create_actor_builder and create_urdf_loader in the ManiskillScene to create some actors and articulations. Actors are created from glb files and articulations created from urdf files. Then I use the function get_axis_aligned_bbox_for_actor and get_axis_aligned_bbox_for_articulation to get bounding boxes. Fucntions are imported from mani_skill.utils.geometry.geometry. But I find that the Actor class and Articulation class are inheritted from Generic[T]. It outputs some error infos as below:

in get_axis_aligned_bbox_for_actor
    for shape in actor.find_component_by_type(
AttributeError: 'Actor' object has no attribute 'find_component_by_type'

in get_axis_aligned_bbox_for_articulation
    for s in link.get_collision_shapes():
AttributeError: 'Link' object has no attribute 'get_collision_shapes'

How can I get the bbox of Actor and Articulation in Maniskill3? By the way, what does the Position attribute represent for a actor or articulation? Is it the centroid or the center of mass of an asset?

[StoneT2000]

Those functions are actually outdated (sorry, we are still removing old / deadcode atm). Currently to get bounding boxes you have to retrieve the collision mesh which returns a trimesh.Mesh object which you can then check with type hints to find all sorts of utilities for meshes. In particular you can get the axis aligned bounding box with this code:

mesh = actor_or_articulation.get_first_collision_mesh() # defaults to a mesh in the world frame
mesh.bounds # shape [2, 3] for low and high values in 3D space in the world frame.

Note this function is called get_first_collision_mesh() since actors/articulation objects are potentially managing multiple objects together. There is also the actor_or_articulation.get_collision_meshes() function which then returns a trimesh.Mesh object for each object managed by actor_or_articulation. e.g. if you have a merged actor object (say a bunch of objects from YCB) you can call get_collision_meshes() and it will return a trimesh.Mesh object for each object in the merged actor.

[DevinJoeUU]

Thanks for your answering. While it seems that not all actor can call function get_first_collision_mesh() or get_collision_meshes() correctly. I tried the command:

python -m mani_skill.examples.demo_random_action -e "ReplicaCAD_SceneManipulation-v1" --render-mode="human"

And I add the following code in the mani_skill/examples/demo_random_action.py file after creating actors

actor = builder.build(name=f"{unique_id}_{actor_name}")
print(f"name:{actor_name}")
# mesh = actor.get_collision_meshes()
mesh = actor.get_first_collision_mesh()
min_bbox, max_bbox = mesh.bounds
center = (min_bbox + max_bbox) / 2
print(f"min_bbox: {min_bbox}")
print(f"max_bbox: {max_bbox}")
print(f"center: {center}")
self._default_object_poses.append((actor, pose))

Then it outputs the following error info:

name:objects/frl_apartment_chair_01-9
name:objects/frl_apartment_rug_02-10
Traceback (most recent call last):
  File "/home/joe/miniconda3/envs/mani3/lib/python3.9/site-packages/gymnasium/envs/registration.py", line 802, in make
    env = env_creator(**env_spec_kwargs)
  File "/home/joe/miniconda3/envs/mani3/lib/python3.9/site-packages/mani_skill/utils/registration.py", line 82, in make
    env = env_spec.make(**kwargs)
  File "/home/joe/miniconda3/envs/mani3/lib/python3.9/site-packages/mani_skill/utils/registration.py", line 35, in make
    return self.cls(**_kwargs)
  File "/home/joe/miniconda3/envs/mani3/lib/python3.9/site-packages/mani_skill/envs/scenes/base_env.py", line 60, in __init__
    super().__init__(*args, robot_uids=robot_uids, **kwargs)
  File "/home/joe/miniconda3/envs/mani3/lib/python3.9/site-packages/mani_skill/envs/sapien_env.py", line 265, in __init__
    obs, _ = self.reset(seed=2022, options=dict(reconfigure=True))
  File "/home/joe/miniconda3/envs/mani3/lib/python3.9/site-packages/mani_skill/envs/scenes/base_env.py", line 88, in reset
    return super().reset(seed, options)
  File "/home/joe/miniconda3/envs/mani3/lib/python3.9/site-packages/mani_skill/envs/sapien_env.py", line 675, in reset
    self._reconfigure(options)
  File "/home/joe/miniconda3/envs/mani3/lib/python3.9/site-packages/mani_skill/envs/sapien_env.py", line 535, in _reconfigure
    self._load_scene(options)
  File "/home/joe/miniconda3/envs/mani3/lib/python3.9/site-packages/mani_skill/envs/scenes/base_env.py", line 97, in _load_scene
    self.scene_builder.build(
  File "/home/joe/miniconda3/envs/mani3/lib/python3.9/site-packages/mani_skill/utils/scene_builder/replicacad/scene_builder.py", line 194, in build
    mesh = actor.get_collision_meshes()
  File "/home/joe/miniconda3/envs/mani3/lib/python3.9/site-packages/mani_skill/utils/structs/actor.py", line 265, in get_collision_meshes
    actor_meshes.append(merge_meshes(get_component_meshes(comp)))
  File "/home/joe/miniconda3/envs/mani3/lib/python3.9/site-packages/mani_skill/utils/geometry/trimesh_utils.py", line 37, in get_component_meshes
    raise TypeError(type(geom))
TypeError: <class 'sapien.pysapien.physx.PhysxCollisionShapeTriangleMesh'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/joe/miniconda3/envs/mani3/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/joe/miniconda3/envs/mani3/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/joe/miniconda3/envs/mani3/lib/python3.9/site-packages/mani_skill/examples/demo_random_action.py", line 98, in <module>
    main(parse_args())
  File "/home/joe/miniconda3/envs/mani3/lib/python3.9/site-packages/mani_skill/examples/demo_random_action.py", line 47, in main
    env: BaseEnv = gym.make(
  File "/home/joe/miniconda3/envs/mani3/lib/python3.9/site-packages/gymnasium/envs/registration.py", line 814, in make
    raise type(e)(
TypeError: <class 'sapien.pysapien.physx.PhysxCollisionShapeTriangleMesh'> was raised from the environment creator for ReplicaCAD_SceneManipulation-v1 with kwargs ({'scene_builder_cls': 'ReplicaCAD', 'obs_mode': 'none', 'reward_mode': None, 'control_mode': None, 'render_mode': 'human', 'shader_dir': 'default', 'sim_backend': 'auto'}

The assets that I used to create actors are mostly from Objaverse. I want to get the bbox of them after loading. Does maniskill3 support getting the bbox of all assets now? Could you give me some advise about it?

[StoneT2000]

Ah I see the bug. It looks like some objects have triangle mesh collision shapes. I will investigate.

However I will point out that most Objaverse objects do not come with collision meshes, just visual meshes. You will have to use something like COACD to decompose those visual meshes into multiple convex collision meshes. This can be done below

builder.add_multiple_convex_collisions_from_file(
  file_path,
  decomposition="coacd",
)

[StoneT2000]

hi @DevinQiao do you have an example .obj / .glb / 3D file I can test with? I will get to fixing the bug soon

[DevinJoeUU]

Sorry for not seeing your message in time. The glb files I use are generally downloaded from Objaverse. The error (https://github.com/haosulab/ManiSkill/issues/404#issuecomment-2213481321) I mentioned uses maniskill3's ReplicaCAD scenario. Do you need a specific asset ID in Objaverse?

[StoneT2000]

Oh I see the file / problem. It can also help if you can send me the .glb file of some asset in Objaverse you want to try that does not work as well.

[DevinJoeUU]

Sure, the following files may help you. The json file includes some object informations. The scale.py is used to normalize the glb file in Objaverse. The remaining two files should be familiar to you. Scene.zip