PSMNet icon indicating copy to clipboard operation
PSMNet copied to clipboard

can not reach your results of val error 1.83 on KITTI15

Open shanyuhu opened this issue 6 years ago • 15 comments

Hi @JiaRenChang I pretrained your stackhourglass network on sceneflow dataset for 10 epoch, and then finetune it on KITTI15 with your provided train/val split for another 300 epoches. However, I can only get 2.8% error rate which is far from your result of 1.83%. All settings are the default ones provided in your code. Did I miss anything? Looking forward to your reply.

shanyuhu avatar Aug 16 '18 04:08 shanyuhu

@shanyuhu Did you check the validation set are those ones? [1,3,6,20,26,35,38,41,43,44,49,60,67,70,81,84,89,97,109,119,122,123,129,130,132,134,141,144,152,158,159,165,171,174,179,182, 184, 186,187,196]

JiaRenChang avatar Aug 16 '18 04:08 JiaRenChang

Yes, I use the same validation set as yours. One difference is that I use larger batch sizes instead of 12 during training. Do you think this will be the reason?

shanyuhu avatar Aug 16 '18 06:08 shanyuhu

By the way, I also use your provided kitti_15 model to evaluate on KITTI15_test. The results are slightly worse than yours in the KITTI leaderboard. don't know why?

shanyuhu avatar Aug 16 '18 07:08 shanyuhu

@shanyuhu We tried batch size = 32,16,12,8,4 in our experiment. We found that the best performance on training scene flow is using batch size = 8 or 12.

It may related to that pytorch updated their upsample function. I am not sure.

JiaRenChang avatar Aug 16 '18 07:08 JiaRenChang

@JiaRenChang thanks for your reply. Btw, whether the provided kitti15 model is the one you used for the final kitti15 leaderboard submission?

shanyuhu avatar Aug 16 '18 11:08 shanyuhu

@shanyuhu That pretrained model is our submission model.

JiaRenChang avatar Aug 19 '18 05:08 JiaRenChang

@JiaRenChang Is the feature extractor in the sceneflow model pre-trained using other dataset, like imagenet?

ghost avatar Sep 14 '18 05:09 ghost

@michael-peng No, it was trained from scratch.

JiaRenChang avatar Sep 14 '18 06:09 JiaRenChang

@shanyuhu Did you check the validation set are those ones? [1,3,6,20,26,35,38,41,43,44,49,60,67,70,81,84,89,97,109,119,122,123,129,130,132,134,141,144,152,158,159,165,171,174,179,182, 184, 186,187,196]

you refer to indexes in [0,199] or [1, 200]? I suppose the first, am I correct?

givasile avatar Oct 19 '18 11:10 givasile

@givasile Yes, [0,199].

JiaRenChang avatar Oct 19 '18 12:10 JiaRenChang

Could you share your finetune PSMNet with me? As my computer's memory is not enough?

lmx-2021 avatar Jan 04 '19 14:01 lmx-2021

Can you tell me how to divide the training set and validate set on KITTI2012, preferably the picture number, thank you!

Chenlei6 avatar Aug 30 '19 11:08 Chenlei6

@Chenlei6 kitti 2012 val set [3,6,20,26,38,41,43,44,49,60,67,70,81,84,89,97,109,119,122,123,129,130,132,134,141,144,152,158,165,171,174,179,184,186]

JiaRenChang avatar Aug 31 '19 11:08 JiaRenChang

@JiaRenChang Thank you!

Chenlei6 avatar Sep 02 '19 12:09 Chenlei6

I‘m sorry to disturb you! Can you share the training method for getting the best model on the KITTI2015 dataset? In the paper , you said that you have trained 1000 epochs. Is there any other training strategy to share?

Chenlei6 avatar Sep 07 '19 12:09 Chenlei6