mscnn
mscnn copied to clipboard
Test dataset accuracy
Hi, @zhaoweicai
May I check whether you have encountered the following problem on the test dataset accuracy? Do you have any suggestions to solve it?
I trained my own network using the KITTI train set and evaluate the accuracy using the val dataset. The way of splitting the two sets are exactly the same as what was described in your paper. I obtained 89% accuracy on the car detection (moderate group). But when I applied the same network on the test dataset and submitted the results to KITTI, the preview page showed only ~75% accuracy for moderate group. For the same network, I also tried to train it using both the train and val dataset and the accuracy on the test dataset was only improved to ~80%, which was still quite bad as compared to the 89% in the val dataset.
Thanks.
Hi @du0002in I agree that the train/val splitting on KITTI is a little bit tricky. However, the splitting I used is kind of reasonable. Although there are some degradation when I validate the model on testing set, I have never experienced ~15% degradation like you have. But degradation like 5~8% could happen I think. It is hard for me to tell what your problems are, since I didn't validate my model on testing too much.
Hi, @du0002in I have enconter the same problem as yours, have you solved the problem?