tune icon indicating copy to clipboard operation
tune copied to clipboard

checks for optimizing the wrong metric type with postptocessing

Open topepo opened this issue 1 year ago • 0 comments

If a post process parameter is marked for tuning, we should check what outcomes it changes and then cross-check with the metrics being measured.

For example, if someone is optimizing the probability cutoff for a binary classification model and they are only measuring ROC AUC (or some other probability-based metric), we should issue a warning since changing the cutoff will not change the metric.

topepo avatar Sep 17 '24 23:09 topepo