timmeinhardt
Track_ids matching problem between different frames
Hello，I found something while training that I don't quite understand
I printed the data preprocessing results about track_ids, the id of the previous frame and the id of the next frame will be misaligned,
` def _getitem_from_id(self, image_id): img, target = super(CocoDetection, self).getitem(image_id) image_id = self.ids[image_id] target = {'image_id': image_id, 'annotations': target}
img, target = self.prepare(img, target)
print("track_ids: ", target['track_ids'])
target['track_ids'] = torch.arange(len(target['labels']))
print("track_ids: ", target['track_ids'])
if self._transforms is not None:
img, target = self._transforms(img, target)
print("track_ids: ", target['track_ids'])`
in an image of 55 frames track_ids: tensor([ 3, 9, 10, 14, 15, 17, 18, 19, 20, 21, 23, 26, 27, 31, 36, 37, 39, 40, 57, 59, 61, 68, 69, 70, 72]) track_ids: tensor([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24]) track_ids: tensor([ 0, 7, 8, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22, 24]) in an image of 57 frames track_ids: tensor([ 2, 3, 9, 10, 14, 15, 17, 18, 19, 20, 21, 23, 26, 27, 31, 36, 39, 40, 57, 59, 61, 68, 69, 70, 72]) track_ids: tensor([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24]) track_ids: tensor([ 1, 8, 9, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22, 24])
Obviously, in 2, 3, 9, 10, 14, 15, 17, 18, 19, 20, 21, 23, 26, 27, 31, 36, two frames are misaligned， 40, 57, 59, 61, 68, 69, 70, 72 are normal
Also I print the result that matches in detr_tracking after
` print("prev_track_ids: ", target['prev_track_ids']) # print("prev_target_ind", prev_target_ind) print("track_ids: ", target['track_ids']) # print(target)
# match track ids between frames
# print(prev_track_ids.unsqueeze(dim=1))
target_ind_match_matrix = prev_track_ids.unsqueeze(dim=1).eq(target['track_ids'])
print(target_ind_match_matrix)
target_ind_matching = target_ind_match_matrix.any(dim=1)
print(target_ind_matching)
target_ind_matched_idx = target_ind_match_matrix.nonzero()[:, 1]
print('track_query_match_ids: ', target_ind_matched_idx)`
prev_track_ids: tensor([ 0, 7, 8, 13, 14, 15, 21, 22, 24], device='cuda:0') track_ids: tensor([ 1, 8, 9, 14, 15, 21, 22, 24], device='cuda:0') tensor([[False, True, False, False, False, False, False, False], [False, False, False, True, False, False, False, False], [False, False, False, False, True, False, False, False], [False, False, False, False, False, False, False, True], [False, False, False, False, False, False, False, False], [False, False, False, False, False, False, False, False], [False, False, False, False, False, True, False, False], [False, False, False, False, False, False, True, False]], device='cuda:0') tensor([ True, True, True, True, False, False, True, True], device='cuda:0') track_query_match_ids: tensor([1, 3, 4, 7, 5, 6], device='cuda:0')
As a result, there seems to be a problem。 So, will there be effects in these places? Thanks。
Hello, I am trying to follow yoru explanation but what do u mean by in an image of 55 frames? In a MOT dataset the track ids will not align in the track_ids list. This is due to objects entering und leaving the scene at different frames.
Also I just fixed a but here https://github.com/timmeinhardt/trackformer/blob/1959ef4c69ca3b418c60beabda352b2647b5726f/src/trackformer/datasets/coco.py#L45
Thank you sir, I think the problem is solved, I always thought that the target_ids of each picture should be rearranged by id