AIF360 icon indicating copy to clipboard operation
AIF360 copied to clipboard

Published 20 hours ago • verified publisher icon Trusted-AI

Include additional classification metrics

Open nrkarthikeyan opened this issue 5 years ago • 5 comments

Include the following metrics:

  1. Equalized odds difference: max(|FPR_unpriv - FPR_priv|, |TPR_unpriv - TPR_priv|)
  2. Generalized equalized odds difference: max(|GFPR_unpriv - GFPR_priv|, |GTPR_unpriv - GTPR_priv|)
  3. Generalized selection rate: mean score possibly conditioned by the group E[\hat{S}]

nrkarthikeyan avatar Jun 04 '19 17:06 nrkarthikeyan

This can be implemented for the "classic": https://github.com/Trusted-AI/AIF360/blob/master/aif360/metrics/classification_metric.py

as well as the sklearn-compatible: https://github.com/Trusted-AI/AIF360/blob/master/aif360/sklearn/metrics/metrics.py

versions

nrkarthikeyan avatar Sep 15 '22 09:09 nrkarthikeyan

Hi! Working on this issue. Can I get assigned to this?

divyagaddipati avatar Sep 22 '23 19:09 divyagaddipati

what does "generalized selection rate" mean? I couldn't find much about it anywhere

divyagaddipati avatar Sep 22 '23 20:09 divyagaddipati

Status:

Classic:

  • [X] Equalized odds difference
  • [X] Generalized equalized odds difference
  • [ ] Generalized selection rate

sklearn:

  • [ ] Equalized odds difference
  • [ ] Generalized equalized odds difference
  • [ ] Generalized selection rate

hoffmansc avatar Oct 11 '23 15:10 hoffmansc

Hi! This PR addresses the equalized odds difference and generalized equalized odds difference for sklearn.

emilymacq avatar Oct 23 '23 14:10 emilymacq