Results for the nonlocal net on Charades dataset

[patykov]

Hi! I'm trying to train the nonlocal net on the Charades dataset and this repo is being of great help! However, I've run your script nonlocal_resnet50_3d_charades.py but it achieved results lower than expected. In the script says that you've reached 32% mAP but my best_model.txt got:

train_loss 0.06904963731537 train_top1 27.30711486067315 train_top5 86.0374044127445 val_loss 0.11540985137626932 val_top1 30.0 val_top5 94.5945945945946 video_task_CharadesmAP 0.17685393374913622 video_task_videotop1 38.16425120772947 video_task_videotop5 147.66505636070855

Is there something else I should be paying attention to run this script?

Also, the paper for this network (https://arxiv.org/pdf/1711.07971.pdf) reports 37.5% mAP. The reason for this gap in the results is known?

[gsig]

Strange, maybe these posts have some information: https://github.com/gsig/PyVideoResearch/issues/13 https://github.com/gsig/PyVideoResearch/issues/11

There is a tiny chance that there has been some code drift since the baseline that would cause these discrepancies, which would be fixed by pulling the version of the repo used to make the baseline. But it's always the possibility that there are differences in the PyTorch/cuda version etc, but it's hard to say.

My only guesses are that batch size is different/number of hours etc. Alternatively it might be worth double checking that the data is all there etc.

To track down what might be going wrong, you can try looking through the log and double check that anything suspicious is at least also present in the log we provide for this baseline.

Hope that helps, Gunnar

On Tue, Sep 10, 2019, 8:44 PM Patrícia Kovaleski [email protected] wrote:

Hi! I'm trying to train the nonlocal net on the Charades dataset and this repo is being of great help! However, I've run your script nonlocal_resnet50_3d_charades.py but it achieved results lower than expected. In the script says that you've reached 32% mAP but my best_model.txt got:

train_loss 0.06904963731537 train_top1 27.30711486067315 train_top5 86.0374044127445 val_loss 0.11540985137626932 val_top1 30.0 val_top5 94.5945945945946 video_task_CharadesmAP 0.17685393374913622 video_task_videotop1 38.16425120772947 video_task_videotop5 147.66505636070855

Is there something else I should be paying attention to run this script?

Also, the paper for this network (https://arxiv.org/pdf/1711.07971.pdf) reports 37.5% mAP. The reason for this gap in the results is known?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/gsig/PyVideoResearch/issues/24?email_source=notifications&email_token=AA3OMNNEMGKPQXNOBBKWM4LQI72JFA5CNFSM4IVLZMAKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HKRG5OQ, or mute the thread https://github.com/notifications/unsubscribe-auth/AA3OMNKXPUL2UKLQQF6AFKLQI72JFANCNFSM4IVLZMAA .