reid_hit_counter not reset when track is matched right before becoming dead

[robbinsa530]

Describe the bug

• TrackedObject to has hit_counter_max=5 and is using RE-ID
• to is initialized and hit_counter=5 (has been tracked continuously for at least 5 frames)
• Tracker.update is called 5 times and to is not matched
  • to.hit_counter is now 0
• Tracker.update is called again
  • to is added to the list of alive_objects per the check in TrackedObject.hit_counter_is_positive (return self.hit_counter >= 0)
  • TrackedObject.tracker_step is called on each tracked object
    • to has its reid_hit_counter set to reid_hit_counter_max because of the >= 0 check on this line.
  • to gets "hit" by a detection in the current frame
  • to has its hit counter incremented back up to 1 (to is alive and well now)
  • When to is hit, its reid_hit_counter is not set back to None. This means that regardless of how it is tracked moving forward (it could be seen in every frame from here on and have a high and healthy hit_counter) it will have its reid_hit_counter decremented on each call to Tracker.update and after reid_hit_counter_max frames, it will be purged due to this code.

To Reproduce

• Create a Tracker with RE-ID enabled
• Create a Detection to be sent to the tracker
• Call Tracker.update with the detection until there is a TrackedObject with a healthy hit_counter
• Call Tracker.update without the detection until the TrackedObject has a hit_counter of 0
• Call Tracker.update with the detection reid_hit_counter_max more times
• The TrackedObject will be deleted

Code:

from norfair import Detection, Tracker
import numpy as np

tracker=Tracker(
    distance_function="euclidean",
    distance_threshold=0.1,
    detection_threshold=0.1,
    initialization_delay=0,
    hit_counter_max=5,
    reid_distance_function="euclidean",
    reid_distance_threshold=0.1,
    reid_hit_counter_max=5,
)

bbox = np.array([[10, 10], [20, 20]])
detection = Detection(
    points=bbox,
    scores=np.array([1.0, 1.0]),
    label=1,
)

# Create healthy track
for i in range(5):
    tracker.update(detections=[detection])

# Print healthy track
for i,to in enumerate(tracker.tracked_objects):
    print(f'TO {i}: ', repr(to), f'(reid_hit_counter: {to.reid_hit_counter})')

# Lose track for hit_counter_max frames
for i in range(5):
    tracker.update(detections=[])

# Print about to be pruned track
for i,to in enumerate(tracker.tracked_objects):
    print(f'TO {i}: ', repr(to), f'(reid_hit_counter: {to.reid_hit_counter})')

# Match track
tracker.update(detections=[detection])

# Show that reid_hit_counter has been activated even though track was matched and hit_counter is positive
for i,to in enumerate(tracker.tracked_objects):
    print(f'TO {i}: ', repr(to), f'(reid_hit_counter: {to.reid_hit_counter})')

# 6 more healthy matches (track considered alive when reid_hit_counter=0, so need 1 extra)
for i in range(6):
    tracker.update(detections=[detection])

# Show that despite maxed out hit_counter, track is about to be pruned due to low reid_hit_counter
for i,to in enumerate(tracker.tracked_objects):
    print(f'TO {i}: ', repr(to), f'(reid_hit_counter: {to.reid_hit_counter})')

# One more update will cause healthy track to be pruned
# NOTE: with initialization_delay=0, if we were to pass `detection` in here,
# the tracker would prune the existing `TrackedObject` and start a new one.
# If we pass nothing, it will simply delete the track
tracker.update(detections=[])

# Show that track has now been pruned
print("Number of tracked objects: ", len(tracker.tracked_objects))

"""
At this point, we would expect the `TrackedObject` to exist with a hit_counter of 4.
Instead it has been pruned
"""

Output:

TO 0:  Object_1(age: 4, hit_counter: 5, last_distance: 0.00, init_id: 1) (reid_hit_counter: None)
TO 0:  Object_1(age: 9, hit_counter: 0, last_distance: 0.00, init_id: 1) (reid_hit_counter: None)
TO 0:  Object_1(age: 10, hit_counter: 1, last_distance: 0.00, init_id: 1) (reid_hit_counter: 5)
TO 0:  Object_1(age: 16, hit_counter: 5, last_distance: 0.00, init_id: 1) (reid_hit_counter: -1)
Number of tracked objects:  0

Expected behavior The TrackedObject should not be deleted

Environment (please complete the following information):

• OS: Mac OS Sonoma 14.3
• Python version: 3.9.19
• Norfair version: 2.2.0

[robbinsa530]

I will try to get a PR out for this

[treygilliland]

Hello @joaqo @dekked @javiber @facundo-lezama! @robbinsa530 discovered a bug relating to the reid_hit_counter while using Norfair. We are big fans of the library at our company and wanted to contribute this change back to the repo. Any chance you could help us out here? We are happy to do the leg work, just need an approval from someone with write access and a new release once the change has been merged

[facundo-lezama]

Hey @robbinsa530 @treygilliland, thanks for reaching out! It's great to hear that you're using and liking Norfair, that's why we love open-source!

Thanks for the detailed explanation and for providing the code to test it. Let me go over it and review the PR.