AdelaiDet icon indicating copy to clipboard operation
AdelaiDet copied to clipboard

train fcpose with coco2017, "AssertionError: assert sum_loss_weight!=0"

Open babyjerry1993 opened this issue 3 years ago • 1 comments

[11/24 15:00:48 adet.trainer]: Starting training from iteration 0 /opt/conda/conda-bld/pytorch_1587428207430/work/torch/csrc/utils/python_arg_parser.cpp:756: UserWarning: This overload of nonzero is deprecated: nonzero(Tensor input, *, Tensor out) Consider using one of the following signatures instead: nonzero(Tensor input, *, bool as_tuple) /opt/conda/conda-bld/pytorch_1587428207430/work/torch/csrc/utils/python_arg_parser.cpp:756: UserWarning: This overload of nonzero is deprecated: nonzero(Tensor input, *, Tensor out) Consider using one of the following signatures instead: nonzero(Tensor input, *, bool as_tuple) /opt/conda/conda-bld/pytorch_1587428207430/work/torch/csrc/utils/python_arg_parser.cpp:756: UserWarning: This overload of nonzero is deprecated: nonzero(Tensor input, *, Tensor out) Consider using one of the following signatures instead: nonzero(Tensor input, *, bool as_tuple) /opt/conda/conda-bld/pytorch_1587428207430/work/torch/csrc/utils/python_arg_parser.cpp:756: UserWarning: This overload of nonzero is deprecated: nonzero(Tensor input, *, Tensor out) Consider using one of the following signatures instead: nonzero(Tensor input, *, bool as_tuple) /root/anaconda3/lib/python3.8/site-packages/torch/nn/functional.py:2970: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details. warnings.warn("Default upsampling behavior when mode={} is changed " /root/anaconda3/lib/python3.8/site-packages/torch/nn/functional.py:2970: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details. warnings.warn("Default upsampling behavior when mode={} is changed " /root/anaconda3/lib/python3.8/site-packages/torch/nn/functional.py:2970: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details. warnings.warn("Default upsampling behavior when mode={} is changed " /root/anaconda3/lib/python3.8/site-packages/torch/nn/functional.py:2970: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details. warnings.warn("Default upsampling behavior when mode={} is changed " Traceback (most recent call last): File "tools/train_net.py", line 229, in launch( File "/root/code/detectron2/detectron2/engine/launch.py", line 55, in launch mp.spawn( File "/root/anaconda3/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 200, in spawn return start_processes(fn, args, nprocs, join, daemon, start_method='spawn') File "/root/anaconda3/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 158, in start_processes while not context.join(): File "/root/anaconda3/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 119, in join raise Exception(msg) Exception:

-- Process 1 terminated with the following error: Traceback (most recent call last): File "/root/anaconda3/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 20, in _wrap fn(i, *args) File "/root/code/detectron2/detectron2/engine/launch.py", line 94, in _distributed_worker main_func(*args) File "/home/xxw/AdelaiDet-master/tools/train_net.py", line 223, in main return trainer.train() File "/home/xxw/AdelaiDet-master/tools/train_net.py", line 102, in train self.train_loop(self.start_iter, self.max_iter) File "/home/xxw/AdelaiDet-master/tools/train_net.py", line 91, in train_loop self.run_step() File "/root/code/detectron2/detectron2/engine/train_loop.py", line 227, in run_step loss_dict = self.model(data) File "/root/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/root/anaconda3/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 445, in forward output = self.module(*inputs[0], **kwargs[0]) File "/root/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/home/xxw/AdelaiDet-master/adet/modeling/one_stage_detector.py", line 46, in forward return super().forward(batched_inputs) File "/root/code/detectron2/detectron2/modeling/meta_arch/rcnn.py", line 285, in forward proposals, proposal_losses = self.proposal_generator(images, features, gt_instances) File "/root/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/home/xxw/AdelaiDet-master/adet/modeling/fcpose/fcpose_framework.py", line 47, in forward detector_results, detector_losses = self.fcpose_head( File "/root/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/home/xxw/AdelaiDet-master/adet/modeling/fcpose/fcpose_head.py", line 231, in forward compute_loss_softmax(gt_bitmasks, larger_mask_logits, File "/home/xxw/AdelaiDet-master/adet/modeling/fcpose/utils.py", line 110, in compute_loss_softmax assert sum_loss_weight!=0 AssertionError

babyjerry1993 avatar Nov 24 '21 15:11 babyjerry1993

what is your batch size?

WeianMao avatar Dec 14 '21 02:12 WeianMao