tune
tune copied to clipboard
checks for optimizing the wrong metric type with postptocessing
If a post process parameter is marked for tuning, we should check what outcomes it changes and then cross-check with the metrics being measured.
For example, if someone is optimizing the probability cutoff for a binary classification model and they are only measuring ROC AUC (or some other probability-based metric), we should issue a warning since changing the cutoff will not change the metric.