Reproducing 2D Only results on the Replica dataset

[shjung13]

Hi Phuc,

Thank you for the great work and deeply appreciate for providing a good code base to start with!

I am trying to run Open3DIS on the Replica dataset (the downsampled dataset you provided along with superpoints), but I could't reproduce your performance in the paper.

======== With code base based on commit 132fb694b9498cee03bf86e4a70bcaa7b68bb97c ========= I am using the following configs, and ran scripts for grounding, generating instances, refining, and generating again to get the final 2D based 3D proposals.

At the end, I ran evaluation with inst_run_replica.py file.

However, I was able to get this number below, which is underperforming your reported performance of 18.1 AP.

Would you let me know if there is any step I need to do for reproducing Replica performance? or Is the performance not reproducible with this version of code?

========= With latest code: 4b05043095aff1dcbc9882799d25e0fb6f4c86a9 ========== I was able to successfully run the grounding part, but I can't run the generate_inst_3d.py file since I don't have post_filter and depth_thresh values that are required for running agglomerative clustering without spp.

Would you provide which values you used for them? + According to configs, you didn't use spp for the Replica dataset.

Is my understanding correct?

Thank you, and looking forward to hearing from you soon!

Best, Sanghun

• Note: I found agglomerative clustering is not runnable for the Replica dataset since it does not return valid confidence (it returns None instead). But this confidence is necessary for getting point-wise CLIP embeddings. Would you also check this?
• I also found several variable errors (visi, num_points, etc) :)