mikropml icon indicating copy to clipboard operation
mikropml copied to clipboard

Precision should be 1 when TP=1 and FP=0

Open kelly-sovacool opened this issue 2 years ago • 0 comments

caret uses MLmetrics::PRAUC(), which reports precision as NA when TP=1 and FP=0. As a result, the prAUC value reported by caret is overinflated because it drops NA values.

Solution: use yardstick::pr_auc(), which uses the convention that precision is 1 when TP=1 and FP=0.

See https://github.com/tidymodels/yardstick/issues/166 & https://github.com/tidymodels/yardstick/pull/95

kelly-sovacool avatar Jun 06 '23 17:06 kelly-sovacool