Error while train when get dataset

[pjy125175]

Hi,When I train the model with python src/train.py with mot17 deformable multi_frame tracking output_dir=models/mot17_deformable_multi_frame, the following errors occur： Traceback (most recent call last): File "src/train.py", line 357, in train(args) File "src/train.py", line 284, in train visualizers['train'], args) File "/home/ubuntu/track/trackformer/src/trackformer/engine.py", line 119, in train_one_epoch for i, (samples, targets) in enumerate(metric_logger.log_every(data_loader, epoch)): File "/home/ubuntu/track/trackformer/src/trackformer/util/misc.py", line 230, in log_every for obj in iterable: File "/home/ubuntu/anaconda3/envs/trackformer/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 345, in next data = self._next_data() File "/home/ubuntu/anaconda3/envs/trackformer/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 856, in _next_data return self._process_data(data) File "/home/ubuntu/anaconda3/envs/trackformer/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 881, in _process_data data.reraise() File "/home/ubuntu/anaconda3/envs/trackformer/lib/python3.7/site-packages/torch/_utils.py", line 395, in reraise raise self.exc_type(msg) IndexError: Caught IndexError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/ubuntu/anaconda3/envs/trackformer/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop data = fetcher.fetch(index) File "/home/ubuntu/anaconda3/envs/trackformer/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/ubuntu/anaconda3/envs/trackformer/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/ubuntu/track/trackformer/src/trackformer/datasets/mot.py", line 49, in getitem img, target = self._getitem_from_id(idx, random_state, random_jitter=False) File "/home/ubuntu/track/trackformer/src/trackformer/datasets/coco.py", line 63, in _getitem_from_id img, target = self.prepare(img, target) File "/home/ubuntu/track/trackformer/src/trackformer/datasets/coco.py", line 220, in call masks = convert_coco_poly_to_mask(segmentations, h, w) File "/home/ubuntu/track/trackformer/src/trackformer/datasets/coco.py", line 177, in convert_coco_poly_to_mask rles = coco_mask.frPyObjects(polygons, height, width) File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects IndexError: list index out of range

I found that the data set is not a pair. But the dataset I generated with this command：python src/generate_coco_from_mot.py.I don't know what's wrong.

[timmeinhardt]

What do you mean by "data set is not a pair"? Did the generate_coco_from_mot.py scripts finish successfully? This looks like a data path issue.

[pjy125175]

Thanks for your answer,I found that the problem was to set the masks of train.yaml to true. I want to generate_ attention_ maps and masks is set to true in track.yaml to evaluate，but I got： RuntimeError: Error(s) in loading state_dict for DeformableDETRSegmTracking: Missing key(s) in state_dict: "bbox_attention.q_linear.weight", "bbox_attention.q_linear.bias", "bbox_attention.k_linear.weight", "bbox_attention.k_linear.bias", "mask_head.lay1.weight", "mask_head.lay1.bias", "mask_head.gn1.weight", "mask_head.gn1.bias", "mask_head.lay2.weight", "mask_head.lay2.bias", "mask_head.gn2.weight", "mask_head.gn2.bias", "mask_head.lay3.weight", "mask_head.lay3.bias", "mask_head.gn3.weight", "mask_head.gn3.bias", "mask_head.lay4.weight", "mask_head.lay4.bias", "mask_head.gn4.weight", "mask_head.gn4.bias", "mask_head.lay5.weight", "mask_head.lay5.bias", "mask_head.gn5.weight", "mask_head.gn5.bias", "mask_head.out_lay.weight", "mask_head.out_lay.bias", "mask_head.adapter1.weight", "mask_head.adapter1.bias", "mask_head.adapter2.weight", "mask_head.adapter2.bias", "mask_head.adapter3.weight", "mask_head.adapter3.bias".

I think this is because the pre-train model does not set masks to true, so I want to set masks to true to retrain my model, and then the problems mentioned at the beginning appear

[timmeinhardt]

Attention maps and mask prediction are two separate things. However, only for our model for mask prediction the attention map plotting is implemented. If you want to generate attention maps for the bbox model, you need to implement attention mask plotting. Not train a model for masks.