Consistently Low Acc for Jester Pretrained models

[gtangg12]

Hi,

The pretrained Jester models give around 50% acc at best. I tried MobileNetV2, MobileNet, and ShuffleNet. Any tips?

Thanks

[cynliu98]

This has happened to me as well, with the default run script for jester and 0.5x MobileNet on the validation set!

[gtangg12]

Someone found the issue a while back:

"I found the mistake that the authors had made that caused me these results. The authors to generate the model invoke the "generate_model" method which always inserts an untrained Linear level as the last level, so I added an "if statement "which in the test case returns the model as it was loaded from the checkpoint. With these changes the performances are similar to those declared by the authors, that is '0.950561' (top5) and '0.624915' on validation. The only consideration is that the results declared by the authors are on the top5 accuracy and not the top1."