Add Prediction Smoothing

[yeldarby]

trafficstars

Description

Adds a Smoother class which averages the last several predictions to decrease noise (like wobbly or flickering boxes).

Before & after smoothing:

https://github.com/roboflow/supervision/assets/870796/160d4d53-51c2-4d82-a6d0-d655fc6eec73

Type of change

• [x] New feature (non-breaking change which adds functionality)

How has this change been tested, please provide a testcase or example of how you tested the change?

Installed locally & tested on several videos/models with InferencePipeline.

Any specific deployment considerations

No.

Docs

• [x] Docs updated? What were the changes: Added new docs page with video & example code.

[SkalskiP]

@yeldarby, please let me know once the PR is ready for the next review round.

[yeldarby]

PR is updated in response to your comments here & @capjamesg's comments on Slack.

[SkalskiP]

@yeldarby I started the second round of review, but I got confused and decided to ask here:

• Is tracker_length still needed? Because it is not used, it is not documented and generally unrelated to the Smoother task (smoothing boxes).
• If tracker_length is not needed, do we need self.track_starts and self.track_ends ? self.track_starts is only there to support tracker_length and self.track_ends is used in two places but the second place can be easily implemented differently.
• If we don't need self.track_starts and self.track_ends we probably do not need self.current_frame.

All around, the whole logic can get a lot easier if we drop tracker_length, which, as I said, is not used; it is not documented and, in general, is unrelated to the Smoother task.

[yeldarby]

Is tracker_length still needed?

I'm using it in some client code to create animated Annotators. The frame of the animation for a detection needs to be tracker_length%num_frames.

Also planning to use it to let people choose if we should wait to display a detection until it's been seen some number of times so that if something is detected in a frame or two but not again it doesn't flicker onto the screen (but haven't implemented that here just yet).

Because it is not used, it is not documented

Good call; if we keep it here I'll document it.

and generally unrelated to the Smoother task (smoothing boxes).

The entry delay will be related to this (though that's easier since you should be able to just take the length of the non-None detections from the tracker; it'd be a tiny bit slower to sum vs reference a count but that's negligible given length will typically be small).

But yeah I think this probably makes a bit more structural sense at the Tracker level (even in my primary use-case you should be able to have an animated Annotator w/o using Smoother); want me to move it there?

[yeldarby]

Updated to remove tracker_length and track_starts.

[capjamesg]

@SkalskiP In my latest change, I add logic to purge unused tracker IDs:

used_tracker_ids = {}

for detection_idx in range(len(detections)):
    tracker_id = detections.tracker_id[detection_idx]
    if tracker_id is None:
        # skip detections without a tracker id
        continue

    if self.tracks.get(tracker_id, None) is None:
        self.tracks[tracker_id] = deque(maxlen=self.length)

    self.tracks[tracker_id].append(detections[detection_idx])
    self.track_ends[tracker_id] = self.current_frame

    used_tracker_ids[tracker_id] = True

for track_id in list(self.tracks.keys()):
    if track_id not in used_tracker_ids:
        del self.tracks[track_id]
        del self.track_ends[track_id]

Here is an example of the smoothing running on a video:

https://github.com/roboflow/supervision/assets/37276661/7f514588-b2c9-4ef0-be91-920156ace99c