allennlp
allennlp copied to clipboard
allenai
Build Fairness Library
Motivation: As models and datasets become increasingly large and complex, it is critical to evaluate the fairness of models according to multiple definitions of fairness and mitigate bias in learned representations. This library aims to make fairness metrics, fairness training tools, and bias mitigation algorithms extremely easy to use and accessible to researchers and practitioners of all levels.
Success Criteria:
- Create a fairness library, and apply it to the Textual Entailment model, publishing an analysis for where the present models fall short and where they should improve.
- Write a blog post and guide chapter and add a model and demo for the implementations of the fairness metrics and bias mitigation algorithms, and explain the broader impact.
Milestones
Implement the following:
Fairness Metrics
- [x] Independence, Separation, Sufficiency
- [x] Sparse Annotations for Ground-Truth
- [ ] Dataset Bias Amplification, Model Bias Amplification
Training-Time Fairness Algorithms (with and without Demographics):
- [x] Through Adversarial Learning (with Demographics)
- [ ] Minimax (without Demographics)
- [ ] Repeated Loss Minimization (without Demographics)
Bias Mitigation Algorithms:
- [x] Linear projection, Hard debiasing, OSCaR, Iterative Null Space Projection
- [x] Bias direction methods: Classification Normal, Two Means, Paired PCA, PCA
- [x] Contextualized word embeddings
Bias Metrics:
- [x] WEAT, Embedding Coherence Test, NLI
Communication:
- [x] blog post
- [x] guide chapter
- [x] demo
- [x] contribute binary gender bias-mitigated model for SNLI to allennlp-models
- [x] contribute binary gender bias-mitigated model for SNLI to demos
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜
@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜