Question about the mIoU on ScanNet validation set

[linhaojia13]

Hi @DylanWusee ,

Thanks for your contribution!

I tried to reproduce your results on ScanNet with default configurations, and there seems a gap between mine and yours.

If I understand your released code correctly, the default configurations is : input-only xyz, evaluation stride-0.5, use Density with MLP. Table 5 in your paper shows that mIoU is 61.0 under this configurations. However, my reproduced result is 0.596368.

The gap is not ignorable, how can I get a better result? I use the default batch_size 8, should I modify this param?

My log_evaluate.txt is as below:

Namespace(batch_size=8, dump_dir='dump', gpu=0, model='pointconv_weight_density_n16', model_path='pointconv_scannet_raw/best_model_epoch_500.ckpt', num_point=8192, num_votes=5, ply_path='../../data/scannet/scans', with_rgb=False) Model restored. 2020-06-08 19:28:12.735647 ---- EVALUATION WHOLE SCENE---- eval point avg class IoU: 0.596368 Each Class IoU::: Class 1 : 0.7410 Class 2 : 0.9473 Class 3 : 0.5330 Class 4 : 0.7170 Class 5 : 0.8281 Class 6 : 0.7131 Class 7 : 0.6537 Class 8 : 0.3413 Class 9 : 0.4584 Class 10 : 0.7243 Class 11 : 0.1304 Class 12 : 0.5907 Class 13 : 0.5174 Class 14 : 0.5421 Class 15 : 0.4015 Class 16 : 0.5021 Class 17 : 0.8202 Class 18 : 0.5903 Class 19 : 0.7632 Class 20 : 0.4123