[Feature] rtmpose Quantization

[mailuo1988]

What is the feature?

The accuracy of the RTMPose model drops significantly when quantized to INT8 for inference, while INT16 inference is acceptable. Are there any solutions?

Any other context?

No response

[shivaa-36]

I was unable to run the quantized RTMpose because mmdeploy was giving an error. I have tried FP16 and int8. but cant run. How were you able to do that.

`loading libmmdeploy_ort_net.so ... [2024-12-09 11:26:50.776] [mmdeploy] [info] [model.cpp:35] [DirectoryModel] Load model: "rtmpose-ort/rtmpose-m-choosen/"

[2024-12-09 11:26:50.776] [mmdeploy] [info] [model.cpp:35] [DirectoryModel] Load model: "rtmpose-ort/rtmdet-nano/"

[2024-12-09 11:26:51.153] [mmdeploy] [info] [inference.cpp:54] ["img"] <- ["data"]

[2024-12-09 11:26:51.153] [mmdeploy] [info] [inference.cpp:65] ["post_output"] -> ["dets"]

[2024-12-09 11:26:51.280] [mmdeploy] [info] [inference.cpp:54] ["img"] <- ["rois"]

[2024-12-09 11:26:51.280] [mmdeploy] [info] [inference.cpp:65] ["post_output"] -> ["keypoints"]

[2024-12-09 11:26:51.998] [mmdeploy] [error] [tensor.cpp:137] mismatched data type FLOAT vs HALF

terminate called without an active exception

Aborted (core dumped)`

@mailuo1988 can you explain the steps you took to run rtmpose fp16

[harry179844]

The static int8 quantization accuracy I'm doing for rtmo has also dropped significantly, with almost all the output confidence levels being 0. But fp16 is acceptable.