DALEX icon indicating copy to clipboard operation
DALEX copied to clipboard

using yardstick (or other package) for performance calculations

Open topepo opened this issue 2 years ago • 0 comments

The loss functions included are great but are somewhat limited.

There are a lot of R packages that could expand the types of loss functions (but at the loss of R:python parity).

If you were to use yardstick, for example, the benefits would be:

  • More metrics
  • Data on direction (e.g larger-is-better) for each metrics. You wouldn't have to do 1 - AUC anymore.
  • yardstick can compute multiple metrics at once.
  • Numerous multi-class metrics
  • Metrics for censored regression
  • Extensible for user-defined metrics

The downside to the current system is that you might optimize your model on a set of performance scores and judge feature importance on some other score.

Let use know if we can help or put in a PR.

topepo avatar Apr 02 '22 16:04 topepo