mmpose
mmpose copied to clipboard
3D Mesh Inference Problem
I have been trying to apply 3D Mesh Inference but the output is nothing like a mesh. The code is:
`import cv2 from mmpose.apis import (inference_top_down_pose_model, inference_mesh_model,init_pose_model, vis_pose_result, process_mmdet_results, vis_3d_mesh_result) from mmdet.apis import inference_detector, init_detector
local_runtime = False
try: from google.colab.patches import cv2_imshow
except: local_runtime = True
pose_config = "hmr_res50_mixed_224x224.py" pose_checkpoint = "hmr_mesh_224x224-c21e8229_20201015.pth" det_config = 'demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py' det_checkpoint = 'https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth'
pose_model = init_pose_model(pose_config, pose_checkpoint, device="cpu") det_model = init_detector(det_config, det_checkpoint, device="cpu")
img = 'tests/data/coco/000000196141.jpg'
mmdet_results = inference_detector(det_model, img)
person_results = process_mmdet_results(mmdet_results, cat_id=1)
pose_results = inference_mesh_model(pose_model,img,person_results)
vis3d_result = vis_3d_mesh_result(pose_model, pose_results, img=img, show=False, out_file="demo/deneme_result2.jpg") vis3d_result = cv2.resize(vis3d_result, dsize=None, fx=0.5, fy=0.5) cv2_imshow(vis3d_result) if local_runtime: from IPython.display import Image, display import tempfile import os.path as osp with tempfile.TemporaryDirectory() as tmpdir: file_name = osp.join(tmpdir, 'pose_results.png') cv2.imwrite(file_name, vis3d_result) display(Image(file_name)) else: cv2_imshow(vis3d_result)`
The output is:
What am I doing wrong here? Isn't there a way that I can get the mesh for given image with just passing config and checkpoint paths?
Thank you
@zengwang430521 Could you please help check this issue?
@codeHorasan BTW, mesh related features will be deprecated in MMPose. We will support it in https://github.com/open-mmlab/mmhuman3d
@codeHorasan Hi, I can get the result like the following:
This may be caused by the error of pyrender. Could you provide your log? Is there any warning in the output?
@zengwang430521 Thank you. Yes, I was getting "pyrender" not installed warning despite it was, i handled it by commenting out pyopengl_environment as I am working on windows. I also had comment out 'img = model' in
`def vis_3d_mesh_result(model, result, img=None, show=False, out_file=None): """Visualize the 3D mesh estimation results.
Args:
model (nn.Module): The loaded model.
result (list[dict]): 3D mesh estimation results.
"""
if hasattr(model, 'module'):
model = model.module
img = model
img = model.show_result(result, img, show=show, out_file=out_file)
return img`
Overall, I am getting the meshes but I was wondering if I have a chance to improve the detection and mesh performance other than changing the models?
You may try discarding some low-confidence bbox.