mmpose
mmpose copied to clipboard
open-mmlab
如何使用rtmo的SDK版onnx模型进行推理
Prerequisite
- [X] I have searched Issues and Discussions but cannot get the expected help.
- [X] The bug has not been fixed in the latest version(https://github.com/open-mmlab/mmpose).
Environment
Python' : '3.8.18 CUDA available' : True PyTorch' : '1.10.0
Reproduces the problem - code sample
我想使用rtmo的SDK版onnx模型进行推理，用mmdeploy中的pose_detection.py进行测试，代码如下
import argparse
import cv2
import numpy as np
from mmdeploy_runtime import PoseDetector
def parse_args():
parser = argparse.ArgumentParser(
description='show how to use sdk python api')
parser.add_argument('device_name', help='name of device, cuda or cpu')
parser.add_argument(
'model_path',
help='path of mmdeploy SDK model dumped by model converter')
parser.add_argument('image_path', help='path of an image')
parser.add_argument(
'--bbox',
default=None,
nargs='+',
type=int,
help='bounding box of an object in format (x, y, w, h)')
args = parser.parse_args()
return args
def main():
args = parse_args()
img = cv2.imread(args.image_path)
detector = PoseDetector(
model_path=args.model_path, device_name=args.device_name, device_id=0)
if args.bbox is None:
result = detector(img)
else:
# converter (x, y, w, h) -> (left, top, right, bottom)
print(args.bbox)
bbox = np.array(args.bbox, dtype=int)
bbox[2:] += bbox[:2]
result = detector(img, bbox)
print(result)
_, point_num, _ = result.shape
points = result[:, :, :2].reshape(point_num, 2)
for [x, y] in points.astype(int):
cv2.circle(img, (x, y), 1, (0, 255, 0), 2)
cv2.imwrite('output_pose.png', img)
if __name__ == '__main__':
main()
Reproduces the problem - command or script
运行命令 python pose_detection.py cpu d:/rtmpose-ort/rtmo d:/1.jpg 进行测试，其中d:/rtmpose-ort/rtmo为sdk目录，但好像无法实现
Reproduces the problem - error message
出现以下信息
loading mmdeploy_trt_net.dll ...
failed to load library mmdeploy_trt_net.dll
loading mmdeploy_ort_net.dll ...
[2024-01-06 15:41:32.149] [mmdeploy] [info] [model.cpp:35] [DirectoryModel] Load model: "d:/rtmpose-ort/rtmo"
[2024-01-06 15:41:32.149] [mmdeploy] [error] [compose.cpp:37] Unable to find Transform creator: BottomupResize. Available transforms: [("CenterCrop", 0), ("Collect", 0), ("Compose", 0), ("DefaultFormatBund
le", 0), ("FormatShape", 0), ("ImageToTensor", 0), ("LetterResize", 0), ("Lift", 0), ("LoadImageFromFile", 0), ("Normalize", 0), ("Pad", 0), ("RescaleToHeight", 0), ("Resize", 0), ("ResizeOCR", 0), ("Short
ScaleAspectJitter", 0), ("TenCrop", 0), ("ThreeCrop", 0), ("TopDownAffine", 0), ("TopDownGetBboxCenterScale", 0)]
[2024-01-06 15:41:32.149] [mmdeploy] [error] [task.cpp:99] error parsing config: {
"context": {
"device": "<any>",
"model": "<any>",
"stream": "<any>"
},
"input": [
"img"
],
"module": "Transform",
"name": "Preprocess",
"output": [
"prep_output"
],
"transforms": [
{
"type": "LoadImageFromFile"
},
{
"input_size": [
640,
640
],
"pad_val": [
114,
114,
114
],
"type": "BottomupResize"
},
{
"mean": [
0,
0,
0
],
"std": [
1,
1,
1
],
"to_rgb": false,
"type": "Normalize"
},
{
"keys": [
"img"
],
"type": "ImageToTensor"
},
{
"keys": [
"img"
],
"meta_keys": [
"img_shape",
"pad_shape",
"ori_shape",
"img_norm_cfg",
"scale_factor",
"bbox_score",
"center",
"scale"
],
"type": "Collect"
}
],
"type": "Task"
}
[2024-01-06 15:41:32.474] [mmdeploy] [error] [common.h:50] Could not found entry 'UNKNOWN' in mmpose. Available components: [("DeepposeRegressionHeadDecode", 0), ("SimCCLabelDecode", 0), ("TopdownHeatmapBa
seHeadDecode", 0), ("TopdownHeatmapMSMUHeadDecode", 0), ("TopdownHeatmapMultiStageHeadDecode", 0), ("TopdownHeatmapSimpleHeadDecode", 0), ("ViPNASHeatmapSimpleHeadDecode", 0)]
[2024-01-06 15:41:32.475] [mmdeploy] [error] [task.cpp:99] error parsing config: {
"component": "UNKNOWN",
"context": {
"device": "<any>",
"model": "<any>",
"stream": "<any>"
},
"input": [
"prep_output",
"infer_output"
],
"module": "mmpose",
"name": "postprocess",
"output": [
"post_output"
],
"params": {
"flip_test": false,
"input_size": [
640,
640
],
"nms_thr": 0.65,
"score_thr": 0.1,
"type": "YOLOXPoseAnnotationProcessor"
},
"type": "Task"
}
Additional information
之前没有用sdk模型处理过one-stage的算法，请问我该怎样才能用rtmo的SDK版onnx模型进行推理呢，十分感谢
Same problem on my side, when converting RTMO to TRT and trying to perform inference with MMDeploy
ubuntu@ubuntu:~/mmdeploy$ python3 tools/deploy.py configs/mmpose/pose-detection_rtmo_tensorrt-fp16_dynamic-640x640.py /home/ubuntu/mmpose/configs/body_2d_keypoint/rtmo/body7/rtmo-t_8xb32-600e_body7-416x416.py https://download.openmmlab.com/mmpose/v1/projects/rtmo/rtmo-t_8xb32-600e_body7-416x416-f48f75cb_20231219.pth /home/ubuntu/mmpose/tests/data/coco/000000000785.jpg --work-dir onnx_dir --dump-info --show --device cuda:0
face the problem
RuntimeError: Unsupported: ONNX export of transpose for tensor of unknown rank.
01/30 08:46:38 - mmengine - ERROR - /home/ubuntu/.local/lib/python3.8/site-packages/mmdeploy/apis/core/pipeline_manager.py - pop_mp_output - 80 - mmdeploy.apis.pytorch2onnx.torch2onnx with Call id: 0 failed. exit.
########################################################
ubuntu@ubuntu:~/mmdeploy$ python3 tools/deploy.py configs/mmpose/pose-detection_rtmo_onnxruntime_dynamic.py /home/ubuntu/mmpose/configs/body_2d_keypoint/rtmo/body7/rtmo-t_8xb32-600e_body7-416x416.py https://download.openmmlab.com/mmpose/v1/projects/rtmo/rtmo-t_8xb32-600e_body7-416x416-f48f75cb_20231219.pth /home/ubuntu/mmpose/tests/data/coco/000000000785.jpg --work-dir onnx_dir --dump-info --show --device cuda:0
face the problem
RuntimeError: Exporting the operator linalg_norm to ONNX opset version 11 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub.
01/30 08:35:59 - mmengine - ERROR - /home/ubuntu/.local/lib/python3.8/site-packages/mmdeploy/apis/core/pipeline_manager.py - pop_mp_output - 80 - mmdeploy.apis.pytorch2onnx.torch2onnx with Call id: 0 failed. exit.
ubuntu@ubuntu:~/mmdeploy$ python3 tools/deploy.py configs/mmpose/pose-detection_rtmo_tensorrt-fp16_dynamic-640x640.py /home/ubuntu/mmpose/configs/body_2d_keypoint/rtmo/body7/rtmo-t_8xb32-600e_body7-416x416.py https://download.openmmlab.com/mmpose/v1/projects/rtmo/rtmo-t_8xb32-600e_body7-416x416-f48f75cb_20231219.pth /home/ubuntu/mmpose/tests/data/coco/000000000785.jpg --work-dir onnx_dir --dump-info --show --device cuda:0
face the problem
RuntimeError: Unsupported: ONNX export of transpose for tensor of unknown rank. 01/30 08:46:38 - mmengine - ERROR - /home/ubuntu/.local/lib/python3.8/site-packages/mmdeploy/apis/core/pipeline_manager.py - pop_mp_output - 80 -
mmdeploy.apis.pytorch2onnx.torch2onnxwith Call id: 0 failed. exit.########################################################
ubuntu@ubuntu:~/mmdeploy$ python3 tools/deploy.py configs/mmpose/pose-detection_rtmo_onnxruntime_dynamic.py /home/ubuntu/mmpose/configs/body_2d_keypoint/rtmo/body7/rtmo-t_8xb32-600e_body7-416x416.py https://download.openmmlab.com/mmpose/v1/projects/rtmo/rtmo-t_8xb32-600e_body7-416x416-f48f75cb_20231219.pth /home/ubuntu/mmpose/tests/data/coco/000000000785.jpg --work-dir onnx_dir --dump-info --show --device cuda:0
face the problem
RuntimeError: Exporting the operator linalg_norm to ONNX opset version 11 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub. 01/30 08:35:59 - mmengine - ERROR - /home/ubuntu/.local/lib/python3.8/site-packages/mmdeploy/apis/core/pipeline_manager.py - pop_mp_output - 80 -
mmdeploy.apis.pytorch2onnx.torch2onnxwith Call id: 0 failed. exit.
also face this problem
