cnn_depth_tensorflow How to train

Hi,

First of all, thanks for this really clean and easy to read code. I tried to train the network using task.py but I have several questions:

are the pictures outputted in the "predict_refine_..." folders really validation pictures? I tested my trained net on other non seen pictures and the results are not as good (actually they are pretty bad).
If I got it right (and checking issue #[1]) the script trains everything together while in the paper the coarse network is trained first before training the fine network (and freezing the coarse predictions). Is that a big issue?
I haven't seen the data augmentation part (from section 3.4 in the paper) in the code, will it impact the results?
Finally, in task.py MAX_STEPS is set to 10000000 but I couldn't run this number of epoch (I went to 4140). How many of them is needed to get good results?

Best,

Clement

Jun 19 '17 12:06 ctiennot

@ctiennot Hey! Also, the paper states that the convolutional layers of the coarse-scale network are pre-trained on the ImageNet classification task [section 3.1.1]. I don't think there is any pre-training going here. Correct me if I am wrong.

Jun 26 '17 05:06 ShrutiGanesh18

@ctiennot , how you tested the model, if possible can you share the script. Were you able to generate good results using this code? Even i am not getting good results.

Aug 11 '17 06:08 kishankedia

@kishankedia, I finally gave up on training the network from this repo and focuses on this repo (the weights to perform inference are provided by the author and explanations are nice).

An important point when training on the NYU depth V2 dataset is that there are two versions of the dataset:

the labeled version (795 training images + about 650 for testing)
the raw version (120k images), which is extracted from the raw kinect sequences, the process being a bit long (following Ivanecky instructions it took me about 3 weeks letting the matlab script running...)

For the eigen CNN they train them on the raw repo with 120k images + data augmetation so I guess that was the reason my training wasn't converging with only 795 images!

Good luck

Aug 11 '17 11:08 ctiennot

@ctiennot Thanks or your quick reply. In this current repo i have tried to do a coarse training , however it is throwing me errors.

Can you suggest me some of the architectures that are currently giving state of the art results for depth and what metrics does the depth models use for comparing different architetures.

Aug 16 '17 08:08 kishankedia

@kishankedia I think this paper is pretty much state of the art, however their weight file is missing. The repo I sent you a link in my last comment(Ivanecky) only provides relative depth estimation (i.e. the predictions are not in meters but between 0 and 1) and it is not state of the art for sure...

About the metrics, there are several ones: for instance there is the absolute error, the scale invariant error or also the thresholded precisions... Ivanecky's thesis has a well explained part on this (here)

Aug 18 '17 09:08 ctiennot

@ctiennot Thanks a lot. Paper looks really good and promising. Unfortunately they have implemented the paper in caffe + matlab. It would have been better if they would implemented in tensorflow.

Aug 18 '17 16:08 kishankedia

@ctiennot @kishankedia @ShrutiGanesh18 I have fixed all the issues from this repo including pre-trained weights and added a validation phase. I have also added two more scales to this network from the new paper. I need some time to open source the work. Every problem is fixed!