Direct Train/Test Accuracy Isuue

[b-hakim]

I trained directly on UCF101 split1. Attached is how the training/validation accuracy plots looks like. As can be seen, the last model weights at step 4999 gives 45% for the validation-set accuracy. When running the prediction script using the weights at step 4999 and calculating the accuracy = sum correct / (sum correct + sum incorrect), I get 26.8% where correct means that the true label is the same as the actual label taken from the prediction script output.

Why is there a difference?

[perfectFeng]

hi I get the same problem,After 10000 steps on UCF101 finetuning,the validation accuracy reduced to 0.45, but I can't find the reason, do you solve it?

[b-hakim]

@perfectFeng During the training, the validation is taken as batches, so the reported accuracy reported during the training is a batch accuracy not the whole validation accuracy. So, the validation actually is 26% and not 45% as reported. In your case, I believe you need to use prediction script and find the accuracy from the resultant .txt file by summing the correct/tot_number.0, I believe 0.45% may be the batch accuracy and not the whole validation dataset accuracy.

[lixun95]

@b-safwat Hello Do we have some ways to get the whole accuracy from prediction script? except counting it by myself

[LiangXu123]

@b-safwat I believe the big margin in validation-set accuracy during training phase and test phase is due to the different data normalization , or due to the Dropout layer, since the dropout and BN layer is only set to TRUE in training pahse