The performance of pretrained model on S3DIS (area1)

[xjdexjt]

Hi, I just use the pretrained model on S3DIS (area1) on my processed dataset using "Stanford3dDataset_v1.2_Aligned_Version" by running following script.

The results are as follows, which is lower that the reported results a lot.

[JonasSchult]

Hi,

I just reran the inference script on a newly processed s3dis dataset. I could reproduce the claimed scores. Did you use the non aligned version of S3DIS?

Moreover, Area 1 contains 44 scenes but your log shows only 23 processed scans. I assume that something happened with your dataset.

Best, Jonas

[xjdexjt]

@JonasSchult Thanks for your reply. I have another problem. I retrain the model on S3DIS. I observe the results of map/ap50/ap25 are close the reported results. But the result (30.1) on mPrec is lower than expected result (68.7 in your paper) a lot.

Should I set the filter_out_instances as True and other hyper-paramters (e.g., scores_threshold)?

Best.

[JonasSchult]

Hi! Yes exactly please set filter_out_instances=true and score_threshold=0.8. We give some more explanations on page 5 in our paper.

Best, Jonas

[peoplelu]

Hello，I want to know which device do you use when you train the model on s3dis？A100？And ，what is the consumption of memory，48gb？Thank you very much.

[JonasSchult]

Hi! I used a single A40 GPU with 48GB RAM.

[peoplelu]

non aligned version of S3DIS

Thank you very much! I also want to know which version of s3dis do you use, non aligned version or aligned?