Training on DDAD dataset

[TienPhuocNguyen]

Thank you for the useful research.

I am trying to train the mono+stereo model with the DDAD dataset (https://github.com/TRI-ML/DDAD). Instead of the rectified stereo images, I used the surround cameras with the known extrinsic parameters, so that I hope the model can learn a correct depth scale. The camera intrinsic params are rescaled and the translations (x,y,z) are multiplied by 10 like training on stereo mode with baseline of 0.1. However, the trained model could only produce meaningful inverse depth but failed to recover the depth scale. Would you mind please letting me know any steps I am missing? Do I need to rescale the camera rotation too?

Thank you.

[daniyar-niantic]

Hi.

The camera rotation should not be rescaled.

For correct depth scale the image pairs need to have sufficient overlap, so if there is no overlap between your stereo pairs then the network would not be able to solve for absolute scale.

Are you using pairs of images from the same timestamp during training?