mmpose icon indicating copy to clipboard operation
mmpose copied to clipboard

Accuracy mismatch between document and test

Open Tau-J opened this issue 3 years ago • 0 comments

Thanks for your error report and we appreciate it a lot. If you feel we have helped you, give us a STAR! :satisfied:

Checklist

  1. I have searched related issues but cannot get the expected help.
  2. The bug has not been fixed in the latest version.

Describe the bug

There's a significant mismatch between document and test result with Deeppose and RLE on MPII

  Head Shoulder Elbow Wrist Hip Knee Ankle PCKh [email protected]
RLE Res50                
Doc 95.80491 94.5822 86.89271 78.34604 87.46754 80.35467 73.45257 86.02394 26.31278
Master 87.82401091405184 94.49728260869566 87.23337152631714 79.61360259379691 87.38096624985977 79.70921574645614 67.7134269811466 84.88420504813948 26.278948799161096
Deeppose Res50                
Doc 93.55389 92.96875 82.73413 73.90828 85.65 75.41952 66.10711 82.53708 20.51002
Master 93.69031377899044 93.01970108695653 82.75121958021805 73.83997186045943 85.54614575872375 75.39928838568132 66.24886621884403 82.53187613843352 17.374446880038818

Reproduction

  • following official tutorial, using res50 ckpt downloaded from document and run tool/test.py

ckpt:

https://download.openmmlab.com/mmpose/top_down/deeppose/deeppose_res50_mpii_256x256_rle-5f92a619_20220504.pth

https://download.openmmlab.com/mmpose/top_down/deeppose/deeppose_res50_mpii_256x256-c63cd0b6_20210203.pth

Environment

torch == 1.12.1+cu113 mmcv-full == 1.6.1

Bug fix

If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!

Tau-J avatar Aug 17 '22 07:08 Tau-J