seismometer icon indicating copy to clipboard operation
seismometer copied to clipboard

Enhance metadata.json to use named thresholds and set threshold bounds

Open gbowlin opened this issue 1 year ago • 0 comments

Is your feature request related to a problem? Please describe

Default score threshold values are read from metadata.json which is loaded from the config.yml's. When loaded, we look at the root thresholds node which is a list of numeric values (unsorted)

This means when we plot model performance, we just sort these high to low, and treat the highest threshold as "Red" and the second highest as "Orange", and we don't provide a name for the threshold like "Telephone Call", "Rapid Response", or "Screening"

This also means that we can only sport one global threshold, even though we could support more than one model output (score)

Describe the solution you'd like

Ideally metadata.json is improved to support different thresholds for different score names (if a model can predict more than one score) and different thresholds for different model outputs.

gbowlin avatar Jul 18 '24 15:07 gbowlin