tri-depth
tri-depth copied to clipboard
xingyuuchen
How can I plug the triplet loss you proposed into my own depth estimation model?
RT，thanks
Hello, @breknddone ,
Thanks for your interests in our work!
To plug the triplet loss into your own model, you can refer to the code comment! The pluging is really simple!
This git commit is about the guidence (I added two lines of comments in the code which you can follow).
Hello, @xingyuuchen , I added the two functions compute_sgt_loss and compute_affinity, but during training, the error is reported as follows:
Traceback (most recent call last):
File "train.py", line 74, in
It seems that this answer cannot be achieved. My data set is not KITTI, but RobotCar and Nuscenes, so I don’t understand how to deal with this situation. Hope to get your answer, thank you!
Hello, @xingyuuchen , I added the two functions compute_sgt_loss and compute_affinity, but during training, the error is reported as follows:
Traceback (most recent call last):
File "train.py", line 74, in
main()File "train.py", line 70, in main
trainer.fit(model, loader)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 609, in fit
self, self._fit_impl, model, train_dataloaders, val_dataloaders, datamodule, ckpt_pathFile "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/trainer/call.py", line 38, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 650, in _fit_impl
self._run(model, ckpt_path=self.ckpt_path)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1112, in _run
results = self._run_stage()File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1191, in _run_stage
self._run_train()File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1214, in _run_train
self.fit_loop.run()File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/loops/loop.py", line 199, in run
self.advance(*args, **kwargs)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/loops/fit_loop.py", line 267, in advance
self._outputs = self.epoch_loop.run(self._data_fetcher)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/loops/loop.py", line 199, in run
self.advance(*args, **kwargs)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/loops/epoch/training_epoch_loop.py", line 213, in advance
batch_output = self.batch_loop.run(kwargs)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/loops/loop.py", line 199, in run
self.advance(*args, **kwargs)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/loops/batch/training_batch_loop.py", line 88, in advance
outputs = self.optimizer_loop.run(optimizers, kwargs)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/loops/loop.py", line 199, in run
self.advance(*args, **kwargs)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 202, in advance
result = self._run_optimization(kwargs, self._optimizers[self.optim_progress.optimizer_position])File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 249, in _run_optimization
self._optimizer_step(optimizer, opt_idx, kwargs.get("batch_idx", 0), closure)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 379, in _optimizer_step
using_lbfgs=is_lbfgs,File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1356, in _call_lightning_module_hook
output = fn(*args, **kwargs)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/core/module.py", line 1754, in optimizer_step
optimizer.step(closure=optimizer_closure)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/core/optimizer.py", line 169, in step
step_output = self._strategy.optimizer_step(self._optimizer, self._optimizer_idx, closure, **kwargs)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/strategies/strategy.py", line 235, in optimizer_step
optimizer, model=model, optimizer_idx=opt_idx, closure=closure, **kwargsFile "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/plugins/precision/precision_plugin.py", line 119, in optimizer_step
return optimizer.step(closure=closure, **kwargs)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/torch/optim/lr_scheduler.py", line 68, in wrapper
return wrapped(*args, **kwargs)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/torch/optim/optimizer.py", line 140, in wrapper
out = func(*args, **kwargs)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/torch/optim/optimizer.py", line 23, in _use_grad
ret = func(self, *args, **kwargs)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/torch/optim/adam.py", line 183, in step
loss = closure()File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/plugins/precision/precision_plugin.py", line 105, in _wrap_closure
closure_result = closure()File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 149, in call
self._result = self.closure(*args, **kwargs)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 135, in closure
step_output = self._step_fn()File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 419, in _training_step
training_step_output = self.trainer._call_strategy_hook("training_step", *kwargs.values())File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1494, in _call_strategy_hook
output = fn(*args, **kwargs)File "/home/user/.conda/envs/rnw/lib/python3.7/site-packages/pytorch_lightning/strategies/strategy.py", line 378, in training_step
return self.model.training_step(*args, **kwargs)File "/home/user/code/RNW/models/mono2.py", line 54, in training_step
disp_loss_dict = self.compute_disp_losses(batch_data, outputs)File "/home/user/code/RNW/models/mono2.py", line 203, in compute_disp_losses
sgt_loss = self.compute_sgt_loss(inputs, outputs)File "/home/user/code/RNW/models/mono2.py", line 88, in compute_sgt_loss
seg_target = inputs['seg', 0, 0]KeyError: ('seg', 0, 0)
It seems that this answer cannot be achieved. My data set is not KITTI, but RobotCar and Nuscenes, so I don’t understand how to deal with this situation.
Hope to get your answer, thank you!
The KeyError is because you tried to load the key that did not exist. You can debug the input loading process step by step using e.g. pycharm, to check at which step your segmentation is not loaded into the form that you requested, i.e. ('seg', 0, 0), or, probably worse, not loaded at all.
The KeyError is because you tried to load the key that did not exist. You can debug the input loading process step by step using e.g. pycharm, to check at which step your segmentation is not loaded into the form that you requested, i.e. ('seg', 0, 0), or, probably worse, not loaded at all.
I checked my code and it was indeed not loaded. How should I modify it?
The KeyError is because you tried to load the key that did not exist. You can debug the input loading process step by step using e.g. pycharm, to check at which step your segmentation is not loaded into the form that you requested, i.e. ('seg', 0, 0), or, probably worse, not loaded at all.
I checked my code and it was indeed not loaded. How should I modify it?
In case that you have other datasets, you must customize the dataloader class, to meet the specific data format of that dataset, like I did in this directory (there are different dataset.py file used for different datasets).