Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sens function giving me the wrong value #494

Open
EduMinsky opened this issue Feb 26, 2024 · 1 comment
Open

Sens function giving me the wrong value #494

EduMinsky opened this issue Feb 26, 2024 · 1 comment
Labels

Comments

@EduMinsky
Copy link

EduMinsky commented Feb 26, 2024

Not sure why (maybe im getting the wrong order in the factors) but R is giving me the wrong values for sensitivity. As you can see in Figure 1, the model detected the true value (Factor Activated) 6 times while the false megative was 14, therefore, my sens should be 0.3 (TP/(TP+FN), but when using summary for the Confusion Matrix (Figure2), it give me 0.742 of sensitivity. Why is R inverting the values?

Figure 1:
my_conf

Figure 2:
summary_conf

Here is a CSV with the actual and predicted values ​​that were used to calculate the tests
MyData.csv

@EmilHvitfeldt
Copy link
Member

Hello @EduMinsky 👋

You are getting these results because by default {yardstick} uses the first factor level (N_Activated in this case) as the basis for calculating metrics. This means that sensibility is calculated as 23/(23 + 8) = 0.742 as we see.

library(yardstick)

my_conf <- structure(list(table = structure(c(23, 8, 14, 6), dim = c(2L, 
  2L), dimnames = list(
    Prediction = c("N_Activated", "Activated"), 
    Truth = c("N_Activated", "Activated")), class = "table")), class = "conf_mat")

my_conf
#>              Truth
#> Prediction    N_Activated Activated
#>   N_Activated          23        14
#>   Activated             8         6

summary(my_conf)
#> # A tibble: 13 × 3
#>    .metric              .estimator .estimate
#>    <chr>                <chr>          <dbl>
#>  1 accuracy             binary        0.569 
#>  2 kap                  binary        0.0443
#>  3 sens                 binary        0.742 
#>  4 spec                 binary        0.3   
#>  5 ppv                  binary        0.622 
#>  6 npv                  binary        0.429 
#>  7 mcc                  binary        0.0459
#>  8 j_index              binary        0.0419
#>  9 bal_accuracy         binary        0.521 
#> 10 detection_prevalence binary        0.725 
#> 11 precision            binary        0.622 
#> 12 recall               binary        0.742 
#> 13 f_meas               binary        0.676

summary(my_conf, event_level = "second")
#> # A tibble: 13 × 3
#>    .metric              .estimator .estimate
#>    <chr>                <chr>          <dbl>
#>  1 accuracy             binary        0.569 
#>  2 kap                  binary        0.0443
#>  3 sens                 binary        0.3   
#>  4 spec                 binary        0.742 
#>  5 ppv                  binary        0.429 
#>  6 npv                  binary        0.622 
#>  7 mcc                  binary        0.0459
#>  8 j_index              binary        0.0419
#>  9 bal_accuracy         binary        0.521 
#> 10 detection_prevalence binary        0.275 
#> 11 precision            binary        0.429 
#> 12 recall               binary        0.3   
#> 13 f_meas               binary        0.353

Created on 2024-02-26 with reprex v2.1.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants