yardstick
yardstick copied to clipboard
tidymodels
Tidy methods for measuring model performance
Hello, Forgive me if this request has already been made but I could not find it. The idea is that the ordinary Brier Score in cases of imbalanced datasets may...
for use as `is_class_metric()` to determine if `accuracy()` is the right thing. We have an `as_tibble()` for metric_sets https://github.com/tidymodels/yardstick/blob/be744a3d419398a165aa5bbea0e7f14e750d5b79/R/aaa-metrics.R#L317 but it is not documented well enough
## The problem For classification metrics, when `truth` or `estimate` is not a factor, `metric_set()` gives a much less informative error than the classification metric itself. ## Reproducible example ```...
[Feature request or question] Provide option for metric analysis by class in a multi-class setting
Not sure if already available easily, but when having a multi-class classification problem, then it would be good to automatically get the performance on several metrics separately for each class,...
In situations with major class imbalance, ROC-AUC may not be a good metric to assess model concordance. Instead, as suggested in numerous places such as the [scikit-learn documentation](https://scikit-learn.org/stable/auto_examples/model_selection/plot_precision_recall.html), the area...
Not sure why (maybe im getting the wrong order in the factors) but R is giving me the wrong values for sensitivity. As you can see in Figure 1, the...
Documentation for `metric_vec_template()` suggests matrix input should be fine. See function documentation: > For matrices, it is best to supply "numeric" as the class to check here. However the code...
Hello. This is my first time here, so I will take the opportunity to first of all thank you for your fantastic package, which is an essential part of my...
## Feature In some situations it might be preferable to pre-specify probability thresholds for the roc curve. Might it be worthwhile to add an argument to `roc_curve` for this?
``` ✖ deprecated-template.R:397: S3 method `multiclass_checks.matrix` needs @export or @exportS3method tag. ✖ [deprecated-template.R:320: S3 method `binary_checks.factor` needs @export or @exportS3method tag. ```