3D Mesh Inference Problem

[codeHorasan]

I have been trying to apply 3D Mesh Inference but the output is nothing like a mesh. The code is:

`import cv2 from mmpose.apis import (inference_top_down_pose_model, inference_mesh_model,init_pose_model, vis_pose_result, process_mmdet_results, vis_3d_mesh_result) from mmdet.apis import inference_detector, init_detector

local_runtime = False

try: from google.colab.patches import cv2_imshow

except: local_runtime = True

pose_config = "hmr_res50_mixed_224x224.py" pose_checkpoint = "hmr_mesh_224x224-c21e8229_20201015.pth" det_config = 'demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py' det_checkpoint = 'https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth'

pose_model = init_pose_model(pose_config, pose_checkpoint, device="cpu") det_model = init_detector(det_config, det_checkpoint, device="cpu")

img = 'tests/data/coco/000000196141.jpg'

mmdet_results = inference_detector(det_model, img)

person_results = process_mmdet_results(mmdet_results, cat_id=1)

pose_results = inference_mesh_model(pose_model,img,person_results)

vis3d_result = vis_3d_mesh_result(pose_model, pose_results, img=img, show=False, out_file="demo/deneme_result2.jpg") vis3d_result = cv2.resize(vis3d_result, dsize=None, fx=0.5, fy=0.5) cv2_imshow(vis3d_result) if local_runtime: from IPython.display import Image, display import tempfile import os.path as osp with tempfile.TemporaryDirectory() as tmpdir: file_name = osp.join(tmpdir, 'pose_results.png') cv2.imwrite(file_name, vis3d_result) display(Image(file_name)) else: cv2_imshow(vis3d_result)`

The output is:

What am I doing wrong here? Isn't there a way that I can get the mesh for given image with just passing config and checkpoint paths?

Thank you

[jin-s13]

@zengwang430521 Could you please help check this issue?

@codeHorasan BTW, mesh related features will be deprecated in MMPose. We will support it in https://github.com/open-mmlab/mmhuman3d

[zengwang430521]

@codeHorasan Hi, I can get the result like the following:

This may be caused by the error of pyrender. Could you provide your log? Is there any warning in the output?

[codeHorasan]

@zengwang430521 Thank you. Yes, I was getting "pyrender" not installed warning despite it was, i handled it by commenting out pyopengl_environment as I am working on windows. I also had comment out 'img = model' in

`def vis_3d_mesh_result(model, result, img=None, show=False, out_file=None): """Visualize the 3D mesh estimation results.

Args:
    model (nn.Module): The loaded model.
    result (list[dict]): 3D mesh estimation results.
"""
if hasattr(model, 'module'):
    model = model.module

img = model
img = model.show_result(result, img, show=show, out_file=out_file)

return img`

Overall, I am getting the meshes but I was wondering if I have a chance to improve the detection and mesh performance other than changing the models?

[jin-s13]

You may try discarding some low-confidence bbox.