temporal-shift-module
temporal-shift-module copied to clipboard
mit-han-lab
Question about finetune on UCF101
I download the pretrained model: TSM_kinetics_RGB_resnet50_shift8_blockres_avg_segment8_e50.pth
and finetune it on UCF101-split1 using the command below:
'''
python main.py ucf101 RGB
--arch resnet50 --num_segments 8
--gd 20 --lr 0.001 --lr_steps 10 20 --epochs 25
--batch-size 64 -j 16 --dropout 0.8 --consensus_type=avg --eval-freq=1
--shift --shift_div=8 --shift_place=blockres
--tune_from=pretrained/TSM_kinetics_RGB_resnet50_shift8_blockres_avg_segment8_e50.pth
'''
but official UCF101 dataset doesn't provide validation dataset, so I split UCF101-split1 into 9:1 , 9 for training , 1 for validation.
after the training process, I test the model on UCF101-split 1 using the command below:
'''
python test_models.py ucf101
--weights=checkpoint/TSM_ucf101_RGB_resnet50_shift8_blockres_avg_segment8_e25/ckpt.best.pth.tar
--test_segments=8 --batch_size=72 -j 24 --test_crops=3 --twice_sample --full_res
'''
and I only get Acc1 93.4%, I want to know what I did wrong, and how can i reproduce your result in the paper Acc1 95.9%. I really appreciate your reply, thank you very much!
Hi, firstly, UCF-101 does provide the official split for training and testing (but there are several splits to prevent overfitting). Could you please try the experiments on split 1 to get the results?
@tonylins Hi, thanks. Do you mean I should train on the whole UCF101 dataset and test on split1?
@Jaychoukobbe No. You should split the dataset into train and eval according to split1.
@tonylins Thanks, but I dont really konw waht you mean. Firstly, I download the three train/test split files from UCF101 dataset official site and as I can see, each train/test split splits the dataset in different way. So what do u mean that "split the dataset into train and eval according to split1"? Does it differ from the original split1 which I download. I really need to reproduce this result, thanks for your help!!
@tonylins And can you share your train/val/test split files with me? Thanks very much.
Hi, for UCF-101, people usually perform experiments on the 3 splits and report the average accuracy to reduce over-fitting. You just need to convert the format of the 3 pairs of (train_list, test_list).
@tonylins @Jaychoukobbe Hi, I also finetune the TSM_kinetics_RGB_resnet50_shift8_blockres_avg_segment8_e50.pth on UCF101 split 1 using the command below: ''' python main.py ucf101 RGB --arch resnet50 --num_segments 8 --gd 20 --lr 0.001 --lr_steps 10 20 --epochs 25 --batch-size 64 -j 16 --dropout 0.8 --consensus_type=avg --eval-freq=1 --shift --shift_div=8 --shift_place=blockres --tune_from=pretrained/TSM_kinetics_RGB_resnet50_shift8_blockres_avg_segment8_e50.pth ''' but I change the batch_size to be 32, and try the experiments using lr 0.001 and 0.0005, finally, I test the model using the command below: ''' python test_models.py ucf101 --weights=checkpoint/TSM_ucf101_RGB_resnet50_shift8_blockres_avg_segment8_e25/ckpt.best.pth.tar --test_segments=8 --batch_size=72 -j 24 --test_crops=3 --twice_sample --full_res ''' but I just get Acc@1 88.84%，it is much worse than 95.9% and 93.4%, so I want to know what I need to do besides that, and if you got Acc@1 95.9% by using rgb&flow fusion. Thanks.
@GoodMan0 did you solve it ? I use TSM_kinetics_RGB_resnet50_shift8_blockres_avg_segment8_e50.pth to finetune UCF101 ,and it only got 87% when training,and also about 87% by using the test_models.py
