yardstick

yardstick copied to clipboard

Feature request: Supporting Brier scores (and decompositions) for two-class outcomes

4

Dear tidymodels team, I wanted to request the addition of Brier scores for measurement of performance for two-class outcome models. The score is described here, along with 2-variable and 3-variable...

kdpsingh

`fall_out()` and `miss_rate()` (i.e. FPR and FNR) to complement `sens()` and `spec()`

1

Not sure if I'm missing something obvious, but I didn't find the false positive rate among the available metrics in yardstick. I know I could simply calculate it by doing...

deschen1

Feature request: calibration metrics

4

I really like `yardstick`'s API for computing discrimination. Calibration is also an important metric for classification models. Is there any interest in developing some `yardstick` functions to compute a calibration...

UAB-BST-680

Lift/gain at a certain percentage

1

I saw an interesting presentation where they optimized the lift statistic but for a fixed percentage of samples that are tested. So for the example data, if someone used `gain_point(two_class_example,...

topepo

Feature request: custom `direction` in `new_metric()`

1

## Feature Apologies if this already exists and I've missed it. It would be useful to allow custom `direction` options when creating new metrics. My current use-case for this behavior...

mikemahoney218

Regression metrics for multivariate models

1

## Feature In situations when you don't have a classification problem but a regression problem with multiple target variables I cannot see a way to make yardstick work with that...

DoktorMike

Add `extract_plot_data()` and fill value to `autoplot()` for type = "mosaic"

4

Resolves part of #240 Just putting this out there to get your thoughts on the interface. I've implemented an `extract_plot_data()` function for the confusion matrix class which returns a list...

joeycouse

Access underlying data of `autoplot()` objects

2

The `autoplot()` functions are great out of the box for quick insights, but I wish there was a way to access the underlying data used to generate said plots. For...

joeycouse

This is a probability-based measure for classification that is an alternative to the area under the ROC curve ([ref](https://link.springer.com/content/pdf/10.1007/s10994-009-5119-5.pdf) and [ref](https://doi.org/10.1002/sim.3859)). There is an implementation [here](https://cran.r-project.org/package=hmeasure) that we can use...

topepo

reportable results metric

1

Can we have a `yardstick` interface to `probably::reportable_rate()`?

topepo