Evaluation on MSR-VTT test set

[hiteshK03]

Hi, following on the above discussion, can you tell how you selected the 2048 samples for both the datasets? Because on calculating FVD for the entire dataset of MSR-VTT i.e. on 2990 videos, I got a score of 328 which is more than the reported value. Therefore, I was curious to know, if I am doing something wrong here.

Thanks.

Originally posted by @hiteshK03 in https://github.com/Doubiiu/DynamiCrafter/issues/6#issuecomment-1893053414

[Doubiiu]

Hi. Sorry for the late reply. I generated 2048 samples (use frame_stride=3) using the 1st frame of 2048 randomly selected videos in MSR-VTT. When computing FVD, please also use frame_stride=3 for the sampled real videos. Please contact me if you have any questions.

[hiteshK03]

Hi, yeah I tried with the above configuration using frame_stride=3, but still got FVD value to be more than 300. Can you also share more about how you calculated FVD, so I can use the same to cross-check. Thanks.