mmpose
                                
                                
                                
                                    mmpose copied to clipboard
                            
                            
                            
                        Accuracy mismatch between document and test
Thanks for your error report and we appreciate it a lot. If you feel we have helped you, give us a STAR! :satisfied:
Checklist
- I have searched related issues but cannot get the expected help.
 - The bug has not been fixed in the latest version.
 
Describe the bug
There's a significant mismatch between document and test result with Deeppose and RLE on MPII
| Head | Shoulder | Elbow | Wrist | Hip | Knee | Ankle | PCKh | [email protected] | |
|---|---|---|---|---|---|---|---|---|---|
| RLE | Res50 | ||||||||
| Doc | 95.80491 | 94.5822 | 86.89271 | 78.34604 | 87.46754 | 80.35467 | 73.45257 | 86.02394 | 26.31278 | 
| Master | 87.82401091405184 | 94.49728260869566 | 87.23337152631714 | 79.61360259379691 | 87.38096624985977 | 79.70921574645614 | 67.7134269811466 | 84.88420504813948 | 26.278948799161096 | 
| Deeppose | Res50 | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Doc | 93.55389 | 92.96875 | 82.73413 | 73.90828 | 85.65 | 75.41952 | 66.10711 | 82.53708 | 20.51002 | 
| Master | 93.69031377899044 | 93.01970108695653 | 82.75121958021805 | 73.83997186045943 | 85.54614575872375 | 75.39928838568132 | 66.24886621884403 | 82.53187613843352 | 17.374446880038818 | 
Reproduction
- following official tutorial, using res50 ckpt downloaded from document and run tool/test.py
 
ckpt:
https://download.openmmlab.com/mmpose/top_down/deeppose/deeppose_res50_mpii_256x256_rle-5f92a619_20220504.pth
https://download.openmmlab.com/mmpose/top_down/deeppose/deeppose_res50_mpii_256x256-c63cd0b6_20210203.pth
Environment
torch == 1.12.1+cu113 mmcv-full == 1.6.1
Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!