about the datasets and reproduce

[QinghuanWei]

Thank you for answering my last question ! Now I have the following new questions：

1. What is the model used to generate pseudo-labels for the kitti dataset? Is it the model given in the model zoo or is it the model trained according to the replication step 2? What is the use of each model given in model zoo, I run the demo.py with 'model_cityscapes_fine' and 'model_kitti_zhou' from the model zoo and it doesn't seem to work the best.
2. which one is the cityscape video sequence dataset, is it leftImg8bit_sequence_trainvaltest.zip?
3. reproduce step 3 'using the Cityscapes-Fine trained model as initialization', is the model used here the one obtained in step 2 or the one in the model zoo.

[markusschoen]

1. The models provided in the model zoo were trained using the commands in the Reproduce Results section. To generate pseudo labels for the KITTI dataset, you can either use the pretrained model_cityscapes_fine.pth model or train the model on Cityscapes following replication step 2. Since Cityscapes has a large training variance, a trained model can have lower performance than the one in the model zoo, which will affect quality of pseudo labels.
2. Yes, the cityscapes video sequence dataset needs leftImg8bit_sequence_trainvaltest.zip.
3. See 1., you can either use the model given in the model zoo or train your own following replication step 2.

[QinghuanWei]

Thank you,I got it, but I generated pseudo labels using the ''cityscapes-fine'' model given in the model zoo according to step 5, and then according to step 6, I used this model as the initialization model with the dataset obtained in step 5 for training. The result is not good, the depth map is very fuzzy, maybe there is something wrong with my operation, the 'total loss' becomes more and more negative during the training process, like the following： "iteration": 19, ……"total_loss": 0.31644707918167114 "iteration": 39, ……"total_loss": -0.3724619448184967 "iteration": 59, ……"total_loss": -0.7168510258197784 ...... "iteration": 59999, ……"total_loss": -4.73533371090889 But, strangely enough, I used the command: python tools/train_net.py --num-gpus 1 --config-file ./configs/MGNet-KITTI-Eigen-Zhou.yaml --eval-only MODEL.WEIGHTS output/2022-08-10_13-19-07_MGNet-KITTI-Eigen-Zhou/model_final.pth for evaluation and I got the result.I have put the results in the attachment. result.zip

[markusschoen]

A negative total loss is normal, it is caused by the uncertainty weighting. Can you post an example of a fuzzy depth map obtained by your model? (You can use the demo script with --output option) Furthermore, have you compared the depth maps of your model with the ones obtained by the model in the model zoo?

[QinghuanWei]

I trained my model by running： python tools/train_net.py --num-gpus 1 --config-file ./configs/MGNet-KITTI-Eigen-Zhou.yaml MODEL.WEIGHTS checkpoint/model_cityscapes_fine.pth I compared the inference results of the model and the results are shown in the attachment. I would also like to ask how to use my own dataset to train the model ？I only train the deep task, I should not need to generate pseudo labels, right? compared.zip

[markusschoen]

Your results are strange, can you please double check your setup? When I run inference on this particular image using the command python tools/demo.py --config-file configs/MGNet-KITTI-Eigen-Zhou.yaml --input "0000000069.png" --output ~/Desktop --opts MODEL.WEIGHTS model_kitti_zhou.pth, I get the following result, which is as smooth as expected:

[markusschoen]

To train on your own dataset, you can follow the instructions for custom datasets in detectron2. Yes, you wont need pseudo labels, if you only train the depth task.