saic_depth_completion icon indicating copy to clipboard operation
saic_depth_completion copied to clipboard

problems about the evaluation

Open chaoer opened this issue 4 years ago • 2 comments

Hi,

Thanks for your great work.

When I tried to test on Mattport3D dataset with config "DM-LRN" and config_file "DM-LRN_efficientnet-b4_pepper.yaml", I can't get similair results as illustrated in README.

The testing environment was created using conda as you wrote. The tesing code is the third line you wrote in Evaluation part.

So I am wondering is there anything I missed. OR are there some tricks should I take when conduct testing ?

P.S. Could you please tell me what's the path of the images used for illustration? Thus, I can test on them and to figure out whether I had created the testing environment correctly.

chaoer avatar Jul 26 '20 13:07 chaoer

Hi,

Thank you for your inspiring work! I downloaded your code and pretrained model weights and tried to evaluate the prediction results on the same Matterport test split as "Deep Depth Completion of a Single RGB-D Image" by Yinda Zhang et al, but unfortunately was unable to reproduce the results in the paper. I tried all models from DM-LRN b2 to DM-LRN b4 but none of results matched what's mentioned in the paper. I am wondering what might happened here - does the depth images need to be preprocessed in a certain way, or is there a different set of model weights available? Thanks in advance for your help!

Best, Yunfan

YunfanZhang42 avatar Aug 02 '20 17:08 YunfanZhang42

Hi, me too. I also tried all the models and the results were not even close to what stated in the paper. I think they had updated something right before they uploaded the code to GitHub (like refactor). And, they accidentally screwed all the settings up. Or, might be, they did some preprocessing but forgot to mention.

tmanh avatar Oct 27 '20 21:10 tmanh