Generalized eta2 and Omega2

[mattansb]

(Originally part of #5 )

https://doi.org/10.1037/1082-989X.8.4.434

[IndrajeetPatil]

Maybe of relevance here: There seems to be an implementation of generalized eta2 in ez (but doesn't have CIs):

https://github.com/mike-lawrence/ez/blob/a7bc677ea35d4cea821fda78958f360c5935144e/R/ezANOVA.R#L57-L72

[IndrajeetPatil]

And there is implementation of generalized omega^2 (and their CIs!) here: https://github.com/doomlab/MOTE/blob/master/R/omega.gen.SS.rm.R

(And detailed post explaining it: https://www.aggieerin.com/shiny-server/tests/gosrmss.html)

[mattansb]

This is the afex code that does this (EtaSq from DescTools doesn't account for observed vars):

if (!is.null(observed) & length(observed) > 0) {
  obs <- rep(FALSE, nrow(tmp2))
  for (i in observed) {
    if (!any(grepl(paste0("\\b", i, "\\b"), rownames(tmp2)))) 
      stop(paste0("Observed variable not in data: ", 
                  i))
    obs <- obs | grepl(paste0("\\b", i, "\\b"), 
                       rownames(tmp2))
  }
  obs_SSn1 <- sum(tmp2$SS * obs)
  obs_SSn2 <- tmp2$SS * obs
}
else {
  obs_SSn1 <- 0
  obs_SSn2 <- 0
}
es_df$ges <- tmp2$SS/(tmp2$SS + sum(unique(tmp2[, "Error SS"])) + 
                        obs_SSn1 - obs_SSn2)

Need to see how to implement this here.

[mattansb]

And there is implementation of generalized omega^2 (and their CIs!) here:

I think this calculation of CIs is somewhat off - the degrees of freedom (den) cannot be the same as those for the partial eta square, they would have to be smaller. Need to look further into this...

[mattansb]

Eta²_G works (slight differences with afex are due to the different method afex uses for computing SSs)

(Still need to add Omega2G)

library(effectsize)

options(contrasts = c('contr.sum', 'contr.poly'))


data(obk.long, package = "afex")
m_afex <- afex::aov_car(value ~ treatment * gender + Error(id),
                        data = obk.long, observed = "gender")

m_aov <- aov(value ~ treatment * gender + Error(id),
             data = m_afex$data$long)


afex::nice(m_afex, es = "ges")
#> Anova Table (Type 3 tests)
#> 
#> Response: value
#>             Effect    df  MSE      F  ges p.value
#> 1        treatment 2, 10 1.52 3.94 + .289    .055
#> 2           gender 1, 10 1.52 3.66 + .189    .085
#> 3 treatment:gender 2, 10 1.52   2.86 .295    .104
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '+' 0.1 ' ' 1

eta_squared(m_aov, generalized = "gender")
#> Group |        Parameter | Eta2 (generalized) |       90% CI
#> ------------------------------------------------------------
#> id    |        treatment |               0.31 | [0.00, 0.58]
#> id    |           gender |               0.14 | [0.00, 0.47]
#> id    | treatment:gender |               0.31 | [0.00, 0.58]

^{Created on 2020-10-09 by the reprex package (v0.3.0)}

[IndrajeetPatil]

Out of curiosity, are you also planning to implement generalized omega^2 in the next release cycle or will it be part of later releases?

I will be coordinating my own package releases accordingly since I am planning to use both of these functions.

[mattansb]

I don't think it'll make it to the next release cycle. I need a break from Omega squared for a while 😩...

[IndrajeetPatil]

lol Okay, cool. I will wait a bit then. :)

[mattansb]

@IndrajeetPatil It is my dream that people use Epsilon squared instead of Omega squared because (1) Epsilon is actually less biased! (2) Calculating Omega is a pain in the a**...

[IndrajeetPatil]

Ah, I see.

My understanding about this is mostly based on Lakens' review of effect sizes: https://www.frontiersin.org/articles/10.3389/fpsyg.2013.00863/full

[mattansb]

Same - but even Lakens is perfect ;)

From: https://doi.org/10.1177/2515245919855053

(Note that adjusted eta is equal to epsilon)

[IndrajeetPatil]

Thanks 👍

I will check it out. I am planning to default to generalized omega-squared in my packages, but maybe I should rethink that.

[mattansb]

I will eventually get to Omega2G (and maybe will try to extrapolate to Epsilon2G? We'll see...)

[DominiqueMakowski]

I also had the same idea about omega > epsilon based on the same source but indeed this is rather compelling evidence

[strengejacke]

Can't unsee... We must add the petit_deux suffix to all columns.

[mattansb]

I'm putting this on ice. Implementing Omega2G is f*^% hard...