scannetv2 leaderboard and validate metric difference

[DurbinLiu]

hello, I trained the model using Scannetv2 dataset, and I got the result AP = 0.453, AP_50 = 0.664, AP_25=0.784. I note that there is a big gap between the APs you reported in the scannetv2 leaderboard and mine. For example, you report AP_25=0.865. so I was confused, Hoping for your reply! best wishes!

[thangvubk]

Your results are on validation or test split?

[DurbinLiu]

Your results are on validation or test split?

The results on validation and 312 scans' test set are similar.

[thangvubk]

For the performance on the validation set. Please refer to this training log. On test set, we train a new model with train + val split. Since ScanNet is small and has performance fluctuation, we visualize to get best epoch.

[DurbinLiu]

@thangvubk Thank you for your reply, my results on the validation are similar to yours in the training log. But I want to know why the performance on the test set improves so many points, could you kindly describe more specifically how to 'visualize to get best epoch' ? Thanks.

[thangvubk]

We just generate the output and visualize it.