ID Switching Issues in ByteTrack with YOLOv8

[durgasaranyu]

I've been using ByteTrack tracker with YOLOv8 for object tracking and have encountered the below ID-switching issues

ID Reassignment for overlapping objects: When one object appears in place of another (occlusion or overlapping), the ID of the first object gets reassigned to the second object, and the first object gets a new ID. ID Switch when objects pass each other: IDs get swapped when two objects pass one another. Current pseudo code for detection + tracking setup:

// initialize the ByteTrack tracker
byte_track::BYTETracker tracker(40, 30);

// preprocess the input frame
cv::Mat preprocessed_frame = preprocess_frame(input_frame);

// run YOLOv8 model for object detection
std::vector<Detections> detections = run_yolov8(preprocessed_frame);

// push the detections to bytetrack objects
std::vector<byte_track::Object> objects;
for (auto &det : detections) {
        // Prepare bounding box and object for tracking
        byte_track::Rect<float> rect(det.x, det.y, det.width, det.height);
        byte_track::Object obj(rect, 0, det.confidence);
        objects.push_back(obj);
}

// update tracker with objects
std::vector<std::shared_ptr<STrack>> tracked_objects = tracker.update(objects);

// Annotate the frame with tracking results
for (const auto &track : tracked_objects) {
    auto rect = track->getRect();
    draw_bounding_box_with_id(input_frame, rect, track->track_id_);
}

// display the annotated frame
return input_frame;`

Tried adjusting the tracker parameters, frame_rate, match_threshold, track_buffer, but did not solve the issue completely.

If anyone has faced similar issues or has any kind of insights into improving the tracking, I'd appreciate your insights. Thank you.

[slantingsun]

@durgasaranyu The code does have issues, but I have not delved into the reasons.

I used emptysoal/TensorRT-YOLOv8-ByteTrack without any problems.

[durgasaranyu]

@slantingsun Thank you for your responce, I tried https://github.com/emptysoal/TensorRT-YOLOv8-ByteTrack/tree/e83089c46abc5de3115c0cbdabbf4c0a06b6e586/bytetrack as suggested but I’m still facing ID-switching issues, especially in cases of occlusions and when objects pass each other.

Despite tweaking the settings, IDs are still being reassigned incorrectly, leading to inconsistencies in tracking. If there are any additional improvements or using a different tracking approach, please suggest. If there is a technique where REID can also be integrated to the bytetrack but without much change in the inference performance, that would also work. I have tried deepsort but dropped it because of decreased performance than bytetrack.

Let me know if there are any specific changes or additional debugging steps that might help resolve this. Thanks!

[fredcombaneyre]

The problem is that there is a bug in byte_track::STrack::predict() , where the prediction is never done. Try to add updateRect(); at the end of this function in STrack.cpp.

This should fix the issue. With this fix it works perfectly fine for me. Hope this helps

[Data-Adventure]

问题是 byte_track::STrack::predict() 中有一个错误，其中预测从未完成。尝试在 STrack.cpp 中此函数的末尾添加 updateRect();。

这应该可以解决问题。修复后，对我来说一切正常。 希望这能有所帮助

you mean add updateRect() func in here?

void byte_track::STrack::predict()
{
    if (state_ != STrackState::Tracked)
    {
        mean_[7] = 0;
    }
    kalman_filter_.predict(mean_, covariance_);
    // updateRect(); // here?
}

[fredcombaneyre]

yes.

[durgasaranyu]

The problem is that there is a bug in byte_track::STrack::predict() , where the prediction is never done. Try to add updateRect(); at the end of this function in STrack.cpp.

This should fix the issue. With this fix it works perfectly fine for me. Hope this helps

I went ahead and added updateRect(); at the end of STrack::predict() in STrack.cpp, but I didn't notice a significant improvement in the results. Initially, I hadn’t made this update because I was already using the actual rectangle coordinates for the bounding boxes, so I didn’t think it would make much of a difference. However, after applying the fix, the issue still seems to persist.

Do you have any other suggestions or insights that might help resolve this?

[diogovazguerra]

Hi @durgasaranyu I had the exact same issue that you have, I tried using other repositories of bytetrack in cpp, but all of them was causing the same issue of ID switching. Well actually it was my fault all along when iterating every tracked object, I was not doing the right way. What I'm try to say is in your code there is a particular function that we can't see when you iterate the updated tracked objects which may be the root of your problem

draw_bounding_box_with_id(input_frame, rect, track->track_id_);

if you show that function on what it does maybe I can help

[junhui-ng]

I encountered the same issue. In my case, the original ByteTrack Python implementation works well, but the C++ implementations (including emptysoal/TensorRT-YOLOv8-ByteTrack) do not provide the same accurate tracking results. I reviewed the code and addressed the discrepancies. You can try my implementation at junhui-ng/ByteTrack-CPP. Hope it helps!

[0xf21]

The problem is that there is a bug in byte_track::STrack::predict() , where the prediction is never done. Try to add updateRect(); at the end of this function in STrack.cpp.

This should fix the issue. With this fix it works perfectly fine for me. Hope this helps

Could you please share the code for updateRect()? Thank you