Different Results on iBims Dataset

[xlDownxl]

Hello everyone,

first of all, thanks for the great work and repository!

I tried to reproduce the results on the iBims dataset using the --ibims flag on the inference script and the original iBims evaluation code. These are the results I am getting with the pretrained 101 model:

rel = 0.18252209 sq_rel = 0.31786945 rms = 1.007896 log10 = 0.093049966 thr1 = 0.67061347 thr2 = 0.8888803 thr3 = 0.953198 dde_0 = 0.8467216 dde_p = 0.014197104 dde_m = 0.1390813 pe_fla = 3.289500754786499 pe_ori = 9.120578994338139 dbe_acc = 2.7233768 dbe_com = 25.422318

The global metrics are as reported in the paper, but for some reason I receive different results for the iBims specific metrics, most notably the DBE com which is 7.03 (or 6.59), while my scripts report 25.422. I am using the same torch and torchvision version as specified in the environment.yaml file. Has anyone an idea on where this difference in results comes from?

Thank You!

[EryiXie]

Ahhh.. I am so sorry that I forget to reply to you. I just tested again and I get the same result as the paper reported. To output .mat format depth with double float accuracy for iBims, there is a specified args "--ibims1" or "--ibims_pd" for it. I assumed if the result you got is very different, could be the case that you use depth output as png file, which is rescaled with a depth_shift factor = 512 to fit into (0, 256x256-1). You can try to run: python simple_inference.py --config=PlaneRecNet_101_config --trained_model=weights/PlaneRecNet_101_9_125000.pth --ibims1=./ibims1/ibims1_core_mat:./ibims1/ibims1_core_mat/test And then use the evaluation script given by iBims1 dataset.