Tracking In YOLO

[SANJAYSGOWDA]

Hello everybody, I am trying to solve the tracking error in the Kalman Filter. The Kalman Filter is good and have almost nicer prediction in the straight road, but when the car is taking a curve (for ref, I am doing tracking on the Pano image, My data is obtained by the Ladybug Camera and I only have Panoramic image, with this I have camera Extrinsic information like (Easting, Northing, Altitude, Yaw, Pitch and Roll of the camera at every location). My Plan is to write a personalized Kalman Filter wherein I can give a correction step , since I know the YPR of the car , I need to corelate it to the shift in the Kalman Filter prediction. I have detection from another custom trained YOLO model (So detection part is solved). I need to just track it, there is alot of Track ID switch. Does anybody have some insights into it.

[UltralyticsAssistant]

👋 Hello @SANJAYSGOWDA, thank you for your interest in Ultralytics 🚀! We recommend starting with a visit to the Docs where you can find detailed guidance and many Python and CLI usage examples. Many of the most common questions may already be addressed there.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to assist us in debugging the issue effectively.

If this is a custom tracking ❓ Question, such as your explanation of customizing a Kalman Filter or handling Track ID switches, please provide as much detail as possible. This can include relevant code snippets, datasets, related parameters, and YOLO outputs. Also, ensure that you are utilizing the latest version of Ultralytics as it may include improvements or updates relevant to your scenario.

Upgrade

To upgrade to the latest ultralytics package and verify that your issue hasn’t already been resolved in the newest version, install it in a Python>=3.8 environment with PyTorch>=1.8:

pip install -U ultralytics

Additional Resources

Feel free to explore our verified environments for running YOLO, which come preloaded with dependencies:

• Notebooks with free GPU:
• Google Cloud Deep Learning VM: GCP Quickstart Guide
• Amazon Deep Learning AMI: AWS Quickstart Guide
• Docker Image: Docker Quickstart Guide

For real-time discussions, join our Discord community 🎧. Prefer in-depth conversation or project insights? Visit Discourse or our Subreddit!

Status

If this badge is green, all Ultralytics CI tests are currently passing. These tests validate operations across all YOLO Modes and Tasks on macOS, Windows, and Ubuntu systems.

This is an automated response 🚀. An Ultralytics engineer will review your issue and provide further assistance soon.

[glenn-jocher]

@SANJAYSGOWDA thank you for sharing the details of your tracking scenario. To address the issues with Track ID switching and improve the Kalman Filter's correction step, you can consider integrating your camera extrinsic information (Yaw, Pitch, Roll) into the state update of the Kalman Filter. This could involve modifying the update and predict functions to account for the motion dynamics influenced by the vehicle's orientation.

The current implementation of the Kalman Filter in Ultralytics, specifically KalmanFilterXYWH, supports customization and could be a good starting point. You can refer to its implementation here: https://docs.ultralytics.com/reference/trackers/utils/kalman_filter/. For insights into extending it, you might need to engineer a transformation matrix that uses your extrinsic data to adjust the state transitions or innovations.

Additionally, incorporating re-identification features or smoothing techniques, as implemented in BOT-SORT (https://docs.ultralytics.com/reference/trackers/bot_sort/), might help reduce Track ID switches. I recommend exploring methods like feature matching or trajectory-based association for enhanced tracking.

Let me know if you have any further questions!

[guguqi123]

@SANJAYSGOWDA thank you for sharing the details of your tracking scenario. To address the issues with Track ID switching and improve the Kalman Filter's correction step, you can consider integrating your camera extrinsic information (Yaw, Pitch, Roll) into the state update of the Kalman Filter. This could involve modifying the update and predict functions to account for the motion dynamics influenced by the vehicle's orientation.

The current implementation of the Kalman Filter in Ultralytics, specifically KalmanFilterXYWH, supports customization and could be a good starting point. You can refer to its implementation here: https://docs.ultralytics.com/reference/trackers/utils/kalman_filter/. For insights into extending it, you might need to engineer a transformation matrix that uses your extrinsic data to adjust the state transitions or innovations.

Additionally, incorporating re-identification features or smoothing techniques, as implemented in BOT-SORT (https://docs.ultralytics.com/reference/trackers/bot_sort/), might help reduce Track ID switches. I recommend exploring methods like feature matching or trajectory-based association for enhanced tracking.

Let me know if you have any further questions!

Hello, may I ask how to only display the ID and not show the class labels when using YOLO's built-in BoT-SORT for tracking?

[glenn-jocher]

@guguqi123 To display only the tracking IDs without class labels when using BoT-SORT, you can modify the visualization settings when plotting the results:

from ultralytics import YOLO

model = YOLO("yolo11n.pt")
results = model.track(source="video.mp4")

# For visualization
for r in results:
    # Set labels=False to hide class labels, but keep boxes and tracking IDs
    annotated_frame = r.plot(labels=False, boxes=True, conf=False)
    # Or customize further using Plot class directly for more control
    # from ultralytics.utils.plotting import Plot
    # p = Plot()
    # annotated_frame = p.plot_trackers(r, labels=False, boxes=True)

You can also customize the tracker's visualization by creating a custom plotting function that renders only the tracking IDs on the bounding boxes.

[github-actions[bot]]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐