mmdeploy icon indicating copy to clipboard operation
mmdeploy copied to clipboard
verified publisher icon open-mmlab

[Bug] MMpose prediction labels only work for single object class

Open lishiyu005 opened this issue 1 year ago • 0 comments

Checklist

  • [X] I have searched related issues but cannot get the expected help.
  • [X] 2. I have read the FAQ documentation but cannot get the expected help.
  • [X] 3. The bug has not been fixed in the latest version.

Describe the bug

mmdeploy/codebase/mmpose/deploy/pose_detection_model.py

def pack_yolox_pose_result(self, preds: List[torch.Tensor], data_samples: List[BaseDataElement]):

line 220: pred_instances.lebels = torch.zeros(bboxes.shape[0])

This line only works for single class label. If prediction instances have more than one class, then the prediction is not correct. And pred_instances.lebels should be changed to pred_instances.labels

Reproduction

from mmdeploy.apis.utils import build_task_processor from mmdeploy.utils import get_input_shape, load_config import torch

deploy_cfg = 'configs/mmpose/pose-detection_onnxruntime_static.py' model_cfg = 'rtmo-l_16xb16-600e-640x640.py' device = 'cpu' backend_model = ['./mmdeploy_models/mmpose/ort/end2end.onnx'] image = './demo/resources/object.jpg'

read deploy_cfg and model_cfg

deploy_cfg, model_cfg = load_config(deploy_cfg, model_cfg)

build task and backend model

task_processor = build_task_processor(model_cfg, deploy_cfg, device) model = task_processor.build_backend_model(backend_model)

process input image

input_shape = get_input_shape(deploy_cfg) model_inputs, _ = task_processor.create_input(image, input_shape)

do model inference

with torch.no_grad(): result = model.test_step(model_inputs)

visualize results

task_processor.visualize( image=image, model=model, result=result[0], window_name='visualize', output_file='output_pose.png')

Environment

- C++ Version: 201703
  - MSVC 192930151
  - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v3.3.2 (Git Hash 2dc95a2ad0841e29db8b22fbccaf3e5da7992b01)
  - OpenMP 2019
  - LAPACK is enabled (usually provided by MKL)
  - CPU capability usage: AVX512
  - CUDA Runtime 12.1
  - NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90
  - CuDNN 8.8.1  (built against CUDA 12.0)
  - Magma 2.5.4
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=8.8.1, CXX_COMPILER=C:/actions-runner/_work/pytorch/pytorch/builder/windows/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /Zc:__cplusplus /bigobj /FS /utf-8 -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE /wd4624 /wd4068 /wd4067 /wd4267 /wd4661 /wd4717 /wd4244 /wd4804 /wd4273, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.2.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 

03/24 23:04:38 - mmengine - INFO - TorchVision: 0.17.0+cu121
03/24 23:04:38 - mmengine - INFO - OpenCV: 4.9.0
03/24 23:04:38 - mmengine - INFO - MMEngine: 0.10.3
03/24 23:04:38 - mmengine - INFO - MMCV: 2.1.0
03/24 23:04:38 - mmengine - INFO - MMCV Compiler: MSVC 193833130
03/24 23:04:38 - mmengine - INFO - MMCV CUDA Compiler: not available
03/24 23:04:38 - mmengine - INFO - MMDeploy: 1.3.1+bc75c9d
03/24 23:04:38 - mmengine - INFO - 

03/24 23:04:38 - mmengine - INFO - **********Backend information**********
03/24 23:04:38 - mmengine - INFO - tensorrt:	None
03/24 23:04:38 - mmengine - INFO - ONNXRuntime:	1.8.1
03/24 23:04:38 - mmengine - INFO - ONNXRuntime-gpu:	1.17.1
03/24 23:04:38 - mmengine - INFO - ONNXRuntime custom ops:	NotAvailable
03/24 23:04:38 - mmengine - INFO - pplnn:	None
03/24 23:04:38 - mmengine - INFO - ncnn:	None
03/24 23:04:38 - mmengine - INFO - snpe:	None
03/24 23:04:38 - mmengine - INFO - openvino:	None
03/24 23:04:38 - mmengine - INFO - torchscript:	2.2.0+cu121
03/24 23:04:38 - mmengine - INFO - torchscript custom ops:	NotAvailable
03/24 23:04:38 - mmengine - INFO - rknn-toolkit:	None
03/24 23:04:38 - mmengine - INFO - rknn-toolkit2:	None
03/24 23:04:38 - mmengine - INFO - ascend:	None
03/24 23:04:38 - mmengine - INFO - coreml:	None
03/24 23:04:38 - mmengine - INFO - tvm:	None
03/24 23:04:38 - mmengine - INFO - vacc:	None
03/24 23:04:38 - mmengine - INFO - 

03/24 23:04:38 - mmengine - INFO - **********Codebase information**********
03/24 23:04:38 - mmengine - INFO - mmdet:	3.3.0
03/24 23:04:38 - mmengine - INFO - mmseg:	None
03/24 23:04:38 - mmengine - INFO - mmpretrain:	1.2.0
03/24 23:04:38 - mmengine - INFO - mmocr:	None
03/24 23:04:38 - mmengine - INFO - mmagic:	None
03/24 23:04:38 - mmengine - INFO - mmdet3d:	None
03/24 23:04:38 - mmengine - INFO - mmpose:	1.3.1
03/24 23:04:38 - mmengine - INFO - mmrotate:	None
03/24 23:04:38 - mmengine - INFO - mmaction:	None
03/24 23:04:38 - mmengine - INFO - mmrazor:	None
03/24 23:04:38 - mmengine - INFO - mmyolo:	None

Error traceback

No response

lishiyu005 avatar Mar 24 '24 22:03 lishiyu005