CLIP4Clip icon indicating copy to clipboard operation
CLIP4Clip copied to clipboard

Published 20 hours ago • verified publisher icon ArrowLuo

Reproduction on LSMDC DataSet

Open JingXiaolun opened this issue 2 years ago • 6 comments

Sorry to disturb you. When I reproduce the results on LSMDC dataset, I get worse results than those in paper. In the meanP experiment, the meanR is always around 200, rather than around 60. The log is shown as follows: 1686898632103 The problem has confused me for a long time and I hope to get some advice for addressing this issue

JingXiaolun avatar Jun 16 '23 06:06 JingXiaolun

@ArrowLuo

JingXiaolun avatar Jun 16 '23 07:06 JingXiaolun

I have observed your training logs and reproduced it. I found that the performance was poor only on the validation set, but returned to normal on the test set. I hope this will be helpful to you

shallowdream66 avatar Jun 27 '23 05:06 shallowdream66

@shallowdream66 ，the test set contains 1000 video clips, and the validation set contains 7408 video clips, do you mean using the 7408 video clips in validation process ?

JingXiaolun avatar Nov 12 '23 05:11 JingXiaolun

Yes，after each epoch it will be verified on the validation set, the checkpoint with the best results will be taken for testing, and the test results will return to normal.

shallowdream66 avatar Nov 12 '23 05:11 shallowdream66

@shallowdream66, thanks a lot, I will try it.

JingXiaolun avatar Nov 12 '23 05:11 JingXiaolun

@shallowdream66 , can you share the training LSMDC log with me ? I still can't reproduce the normal results in the test set

JingXiaolun avatar Nov 12 '23 07:11 JingXiaolun