GroundNLQ icon indicating copy to clipboard operation
GroundNLQ copied to clipboard

Fail to reproduce the "Training-From-Scratch" performance

Open Becomebright opened this issue 1 year ago • 2 comments

Thanks for your impressive work.

I try to reproduce the "Training-From-Scratch" performance using: bash tools/train_ego4d_twogpu.sh configs/ego4d_nlq_v2_internvideo_1e-4.yaml scratch_2gpu 0,1. The evaluation result is about 14-15 R@1, IoU=0.3 on the validation set, significantly lower than the reported number. Could you please give me some advice on reproducing the "Training-From-Scratch" performance?

First, there's a typo in README. I think the "Training-From-Scratch" command should be bash tools/train_ego4d_twogpu.sh configs/ego4d_nlq_v2_internvideo_1e-4.yaml scratch_2gpu 0,1.

Moreover, trunc_thresh and crop_ratio in ego4d_nlq_v2_internvideo_1e-4.yaml have to be deleted.

Further, the learning_rate in ego4d_nlq_v2_internvideo_1e-4.yaml is 5e-5, contradicting with the config name.

Could you please check these potential mistakes and provide a reproducible config?

Becomebright avatar Jul 11 '23 04:07 Becomebright