supervision
supervision copied to clipboard
roboflow
Cost Matrix contains NaN in ByteTrack() code
Search before asking
- [X] I have searched the Supervision issues and found no similar bug report.
Bug
I got the following error message when using supervision ByteTrack(). Something in the image is causing a NaN value to appear in the cost matrix in matching.py. I have got around the issue for now by inserting the below line of code in bold to remove the NaN value from the matrix in line 32 of matching.py:
cost_matrix[cost_matrix > thresh] = thresh + 1e-4
**cost_matrix = np.nan_to_num(cost_matrix)**
row_ind, col_ind = linear_sum_assignment(cost_matrix)
indices = np.column_stack((row_ind, col_ind))
console output:
/Users/tobieabel/PycharmProjects/Demo_4_Fight_and_Fall/venv/lib/python3.11/site-packages/supervision/detection/utils.py:56: RuntimeWarning: invalid value encountered in divide
return area_inter / (area_true[:, None] + area_detection - area_inter)
Traceback (most recent call last):
File "/Users/tobieabel/PycharmProjects/Demo_4_Fight_and_Fall/Main.py", line 192, in
Environment
- Supervision v0.15.0
- Python 3.11
Minimal Reproducible Example
I'm using the following code to run YOLO8 object detector through supervision to plot the tracking on the annotated frame
def Run_Models(self,frame:np.ndarray) ->sv.detection: obj_det_result = self.object_det_model.predict(frame, device = "mps", verbose = False, conf=0.4,iou=0.7)[0]#send every frame to YOLO model obj_detections = sv.Detections.from_ultralytics(obj_det_result) # pass the results from the model to the sv libary as they are easier to manipulate obj_detections= obj_detections[obj_detections.class_id == 0] # filter the list of detections so it only shows category '0' which is people obj_detections = self.tracker.update_with_detections(obj_detections) # pass the detections through the tracker to add tracker ID as additional field to detections object
Additional
No response
Are you willing to submit a PR?
- [ ] Yes I'd like to help by submitting a PR!
Hello there, thank you for opening an Issue ! 🙏🏻 The team was notified and they will get back to you asap.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Hi @tobieabel 👋🏻 Thanks for reporting the issue. Is it possible for you to provide us with a video example that we could use for debugging? I mean the video that is causing that issue for you.
Sorry for the late reply, the video is attached, but I think the issue may have been caused by the object detector I was using at the time, which was the standard YOLOv8n model. Once I retrained it to get better detection results the issue seemed to go away.
https://github.com/roboflow/supervision/assets/34671814/8d7cf557-f70c-4f5d-9a0d-867bad690b59
no, it was only that one time. When I use a better performing model the tracking doesn't suffer from that problem anymore
@rolson24, do you have any idea what could be the cause of this old issue?
Hi @SkalskiP Based on the warning at the beginning of the console output:
/Users/tobieabel/PycharmProjects/Demo_4_Fight_and_Fall/venv/lib/python3.11/site-packages/supervision/detection/utils.py:56: RuntimeWarning: invalid value encountered in divide
return area_inter / (area_true[:, None] + area_detection - area_inter)
The only thing I can think of is that the bounding boxes coming from the YOLO model were either NaN, or they were bounding boxes that had 0 area. I'm leaning towards the boxes likely having 0 area (either the width is 0 or the height is 0) because I doubt Ultralytics would return boxes with null coordinates, but they might have returned boxes with 0 area.
I think the best option to possibly fix this is to simply set all of the NaN values to a cost value above the match threshold in the linear_assignment function. This would keep all of the detections around, but just not match any of the 0 area detections with any existing tracks. Then these 0 area detections, could get initialized as new tracks. Alternatively, we could just filter them right away at the beginning of update_with_tensors by calculating the area of each bounding box and then filtering out the ones with 0 area.
Thanks @rolson24 🙏🏻 I think that bounding boxes with zero area are the most likely scenario. Would you be interested in writing a PR that would protect us from this?
