dive icon indicating copy to clipboard operation
dive copied to clipboard

Add method for displaying an adjusted uncertainty range for each type count

Open mattdawkins opened this issue 3 years ago • 5 comments
trafficstars

Is your feature request related to a problem? If so, Please describe.

All detectors have false alarms and missed detections, we need some way to display either incorporated uncertainty measures into final counts (which might produce a range and range confidence) or a fix scaling of the counts for each individual type (which might typically be just one value, a scaled count) for a sequence for counting purposes.

Describe the solution you'd like

A panel or some extension of the current types panel which displays adjusted type counts, per type/object category.

Adjusted type counts might usually be some function of the number of detections of a specified type in an image above some threshold, summed across all frames and rounded off. Occasionally there might be 2 functions for a lower and maximum bound as well producing a range and uncertainty estimate

.

mattdawkins avatar Mar 29 '22 14:03 mattdawkins

Please provide more information about the data. What data, in what format, would a pipeline produce to indicate these uncertainties? If this is a new file or a change to the CSV, an example would be helpful

subdavis avatar Mar 29 '22 14:03 subdavis

These will not be produced by a pipeline, it will be an external function determined after the fact in offline tools.

Ideally we'd want this stored in an importable configuration that contains both the thresholds these scaling functions are meant for and were computed at (using our scoring tools), and the per-category thresholds.

In most cases the 'function' will be a linear scaling of the 2 input variables, but (possibly) not always. We still need to determine what it is over the next month. A likely first pass equation for development sake would new_count = round( for every frame( sX + c ) ) where s is some scale factor and c some constant, and X the number of detentions of that type in the image.

mattdawkins avatar Mar 29 '22 14:03 mattdawkins

The next level up would just be a piecewise version of the above

new_count = round( for every frame( ( s1X + c1 ) if X < 10 else s2X + c2 ) )

mattdawkins avatar Mar 29 '22 14:03 mattdawkins

@mattdawkins, can you provide us with an example of how we would get these values within DIVE? That is, can you send us a VIAME CSV, along with an example output detailing the s, c, and "threshold" values corresponding to it? That will help us understand how to engineer a solution for this issue.

waxlamp avatar Apr 19 '22 19:04 waxlamp

We're still deciding what that's going to be but hopefully will over the next week (or two)

mattdawkins avatar Apr 21 '22 13:04 mattdawkins