responsible-ai-toolbox icon indicating copy to clipboard operation
responsible-ai-toolbox copied to clipboard

Enable cohort filtering for "Predicted Y"

Open gaugup opened this issue 3 years ago • 6 comments

Description

This PR enables cohort filtering capability for "Predicted Y". It enables this by adding the predicted values to the dataframe before the cohort filtering logic using the pandas query function. This PR has the following changes:-

  1. Add predicted y values in all cases in cohort_filter.py. If the predictions are available apriori then it adds those predictions.
  2. Adds tests in test_cohort_filter.py to run tests from PredictionAnalyzer which takes predictions instead of the model.
  3. Enables cohort filtering for Predicted Y case.
  4. Other changes in erroranalysis package to handle scenario when predicted y is present on the filtered data.

Checklist

  • [ ] I have added screenshots above for all UI changes.
  • [ ] I have added e2e tests for all UI changes.
  • [ ] Documentation was updated if it was needed.

gaugup avatar Sep 15 '22 21:09 gaugup

Codecov Report

Merging #1722 (71bfc81) into main (32408d5) will decrease coverage by 0.68%. The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #1722      +/-   ##
==========================================
- Coverage   92.52%   91.84%   -0.68%     
==========================================
  Files          55       35      -20     
  Lines        2127      699    -1428     
==========================================
- Hits         1968      642    -1326     
+ Misses        159       57     -102     
Flag Coverage Δ
unittests 91.84% <ø> (-0.68%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

see 20 files with indirect coverage changes

codecov-commenter avatar Sep 15 '22 21:09 codecov-commenter

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/gaugup/AddPredYCohortFiltering/dashboard/index.html

github-actions[bot] avatar Sep 15 '22 22:09 github-actions[bot]

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/gaugup/AddPredYCohortFiltering/dashboard/index.html

github-actions[bot] avatar Sep 15 '22 22:09 github-actions[bot]

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/gaugup/AddPredYCohortFiltering/dashboard/index.html

github-actions[bot] avatar Jun 21 '23 23:06 github-actions[bot]

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/gaugup/AddPredYCohortFiltering/dashboard/index.html

github-actions[bot] avatar Jun 21 '23 23:06 github-actions[bot]

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/gaugup/AddPredYCohortFiltering/dashboard/index.html

github-actions[bot] avatar Jun 26 '23 19:06 github-actions[bot]