mmdeploy [Bug] - Support Mask2Former in C++ SDK

trafficstars

Checklist

[X] I have searched related issues but cannot get the expected help.
[X] 2. I have read the FAQ documentation but cannot get the expected help.
[X] 3. The bug has not been fixed in the latest version.

Describe the bug

Hi, I successfully exported mask2former model with ONNX backend but when I try to use it with the C++ SDK, I get an error.

The error is localized in the object_detection.cpp file in the ResizeBBox::GetDetsLabels function. It is due to the fact that the function tries to search the name "dets", "labels" or "detection_output" as key of inference results container, but for panoptic segmentation those keys are cls_logits and mask_logits, as also reported in the detail.json file of the exported model.

Since the support of panoptic segmentation model deployment was already merged in the current branch (1.3.1), is the problem that the C++ SDK was not updated yet to support the panoptic model, or there are some modification to adopt to make it work?

Thank, you __

Reproduction

I used the detector.cxx main to run the model.

Environment

01/15 15:40:02 - mmengine - INFO - 

01/15 15:40:02 - mmengine - INFO - **********Environmental information**********
01/15 15:40:03 - mmengine - INFO - sys.platform: linux
01/15 15:40:03 - mmengine - INFO - Python: 3.8.16 (default, Mar  2 2023, 03:21:46) [GCC 11.2.0]
01/15 15:40:03 - mmengine - INFO - CUDA available: True
01/15 15:40:03 - mmengine - INFO - numpy_random_seed: 2147483648
01/15 15:40:03 - mmengine - INFO - GPU 0: NVIDIA RTX A4000
01/15 15:40:03 - mmengine - INFO - CUDA_HOME: /usr/local/cuda
01/15 15:40:03 - mmengine - INFO - NVCC: Cuda compilation tools, release 11.7, V11.7.99
01/15 15:40:03 - mmengine - INFO - GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
01/15 15:40:03 - mmengine - INFO - PyTorch: 1.13.1
01/15 15:40:03 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - GCC 9.3
  - C++ Version: 201402
  - Intel(R) oneAPI Math Kernel Library Version 2023.1-Product Build 20230303 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.7
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.5
  - Magma 2.6.1
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.7, CUDNN_VERSION=8.5.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.13.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, 

01/15 15:40:03 - mmengine - INFO - TorchVision: 0.14.1
01/15 15:40:03 - mmengine - INFO - OpenCV: 4.7.0
01/15 15:40:03 - mmengine - INFO - MMEngine: 0.8.5
01/15 15:40:03 - mmengine - INFO - MMCV: 2.0.0
01/15 15:40:03 - mmengine - INFO - MMCV Compiler: GCC 9.3
01/15 15:40:03 - mmengine - INFO - MMCV CUDA Compiler: 11.7
01/15 15:40:03 - mmengine - INFO - MMDeploy: 1.3.1+
01/15 15:40:03 - mmengine - INFO - 

01/15 15:40:03 - mmengine - INFO - **********Backend information**********
01/15 15:40:03 - mmengine - INFO - tensorrt:    8.6.0
01/15 15:40:03 - mmengine - INFO - tensorrt custom ops: Available
01/15 15:40:03 - mmengine - INFO - ONNXRuntime: None
01/15 15:40:03 - mmengine - INFO - ONNXRuntime-gpu:     1.12.0
01/15 15:40:03 - mmengine - INFO - ONNXRuntime custom ops:      Available
01/15 15:40:03 - mmengine - INFO - pplnn:       None
01/15 15:40:03 - mmengine - INFO - ncnn:        None
01/15 15:40:03 - mmengine - INFO - snpe:        None
01/15 15:40:03 - mmengine - INFO - openvino:    None
01/15 15:40:03 - mmengine - INFO - torchscript: 1.13.1
01/15 15:40:03 - mmengine - INFO - torchscript custom ops:      NotAvailable
01/15 15:40:03 - mmengine - INFO - rknn-toolkit:        None
01/15 15:40:03 - mmengine - INFO - rknn-toolkit2:       None
01/15 15:40:03 - mmengine - INFO - ascend:      None
01/15 15:40:03 - mmengine - INFO - coreml:      None
01/15 15:40:03 - mmengine - INFO - tvm: None
01/15 15:40:03 - mmengine - INFO - vacc:        None
01/15 15:40:03 - mmengine - INFO - 

01/15 15:40:03 - mmengine - INFO - **********Codebase information**********
01/15 15:40:03 - mmengine - INFO - mmdet:       3.3.0
01/15 15:40:03 - mmengine - INFO - mmseg:       None
01/15 15:40:03 - mmengine - INFO - mmpretrain:  1.0.2
01/15 15:40:03 - mmengine - INFO - mmocr:       None
01/15 15:40:03 - mmengine - INFO - mmagic:      None
01/15 15:40:03 - mmengine - INFO - mmdet3d:     None
01/15 15:40:03 - mmengine - INFO - mmpose:      None
01/15 15:40:03 - mmengine - INFO - mmrotate:    None
01/15 15:40:03 - mmengine - INFO - mmaction:    None
01/15 15:40:03 - mmengine - INFO - mmrazor:     None
01/15 15:40:03 - mmengine - INFO - mmyolo:      None

Error traceback

[2024-01-15 14:54:52.345] [mmdeploy] [info] [zip_model_impl.cpp:41] Open model file /software/projects/panoptic_onnx.zip successfully
[2024-01-15 14:54:52.346] [mmdeploy] [info] [zip_model_impl.cpp:115] There are 4 files in the model
[2024-01-15 14:54:52.346] [mmdeploy] [info] [model.cpp:35] [ZipModel] Load model: "/software/projects/panoptic_onnx.zip"
[2024-01-15 15:27:48.913] [mmdeploy] [error] [object_detection.cpp:64] No support for another key of detection results!