AiOS icon indicating copy to clipboard operation
AiOS copied to clipboard
verified publisher icon ttxskk

The evaluation results differ from the paper tables

Open liaowang0125 opened this issue 1 year ago • 2 comments

I run the agora val evaluation scripts, but there are many differences between the results and the paper, evaluation results in the screenshot below, only the MVE and MPJPE are similar with paper, NMVE and NMJE are much lower than the paper.What is the reason for this? img_v3_02gi_e9687a6c-b679-4859-ade3-24f2bc82108g

liaowang0125 avatar Nov 12 '24 08:11 liaowang0125

The precision of your test result is 0.28, so I guess you set a lower value for the threshold. If you use new checkpoints, try adjusting the score threshold.

ttxskk avatar Jan 22 '25 06:01 ttxskk

@liaowang0125 @ttxskk hello, can you tell me how to get the results? by agora_evaluation tool or by this command sh scripts/test_agora_val.sh data/checkpoint/aios_checkpoint.pth agora_val? Did you use agora_evaluation tool?if you use, could you tell me the command? if you do not use agora_evalution tool, could you tell me why i can not get the results like you above?

Image

shuaiqidenanhai avatar Apr 05 '25 07:04 shuaiqidenanhai