yardstick icon indicating copy to clipboard operation
yardstick copied to clipboard

Sens function giving me the wrong value

Open EduMinsky opened this issue 11 months ago • 1 comments

Not sure why (maybe im getting the wrong order in the factors) but R is giving me the wrong values for sensitivity. As you can see in Figure 1, the model detected the true value (Factor Activated) 6 times while the false megative was 14, therefore, my sens should be 0.3 (TP/(TP+FN), but when using summary for the Confusion Matrix (Figure2), it give me 0.742 of sensitivity. Why is R inverting the values?

Figure 1: my_conf

Figure 2: summary_conf

Here is a CSV with the actual and predicted values ​​that were used to calculate the tests MyData.csv

EduMinsky avatar Feb 26 '24 12:02 EduMinsky

Hello @EduMinsky 👋

You are getting these results because by default {yardstick} uses the first factor level (N_Activated in this case) as the basis for calculating metrics. This means that sensibility is calculated as 23/(23 + 8) = 0.742 as we see.

library(yardstick)

my_conf <- structure(list(table = structure(c(23, 8, 14, 6), dim = c(2L, 
  2L), dimnames = list(
    Prediction = c("N_Activated", "Activated"), 
    Truth = c("N_Activated", "Activated")), class = "table")), class = "conf_mat")

my_conf
#>              Truth
#> Prediction    N_Activated Activated
#>   N_Activated          23        14
#>   Activated             8         6

summary(my_conf)
#> # A tibble: 13 × 3
#>    .metric              .estimator .estimate
#>    <chr>                <chr>          <dbl>
#>  1 accuracy             binary        0.569 
#>  2 kap                  binary        0.0443
#>  3 sens                 binary        0.742 
#>  4 spec                 binary        0.3   
#>  5 ppv                  binary        0.622 
#>  6 npv                  binary        0.429 
#>  7 mcc                  binary        0.0459
#>  8 j_index              binary        0.0419
#>  9 bal_accuracy         binary        0.521 
#> 10 detection_prevalence binary        0.725 
#> 11 precision            binary        0.622 
#> 12 recall               binary        0.742 
#> 13 f_meas               binary        0.676

summary(my_conf, event_level = "second")
#> # A tibble: 13 × 3
#>    .metric              .estimator .estimate
#>    <chr>                <chr>          <dbl>
#>  1 accuracy             binary        0.569 
#>  2 kap                  binary        0.0443
#>  3 sens                 binary        0.3   
#>  4 spec                 binary        0.742 
#>  5 ppv                  binary        0.429 
#>  6 npv                  binary        0.622 
#>  7 mcc                  binary        0.0459
#>  8 j_index              binary        0.0419
#>  9 bal_accuracy         binary        0.521 
#> 10 detection_prevalence binary        0.275 
#> 11 precision            binary        0.429 
#> 12 recall               binary        0.3   
#> 13 f_meas               binary        0.353

Created on 2024-02-26 with reprex v2.1.0

EmilHvitfeldt avatar Feb 26 '24 17:02 EmilHvitfeldt