Oliver Thomas

Bibtex for datasets and models

1

Every model should have a copy of the bibtex in the docs. Ideally same for the datasets, and I guess any metrics too if appropriate.

Confusion matrix values are often used in metrics, but some people might want these values recording directly. A user story would be something like Given: everything is installed When: I...

The Aequitas tool is really cool. It would be great to have something similar

implementation: https://github.com/mlohaus/SearchFair

so we're not completely bombarded, here's results for the first few files of `ethicml.metrics` ## accuracy.py ```diff ---- ethicml/metrics/accuracy.py (6) ---- [3/1889] # mutant 6500 --- ethicml/metrics/accuracy.py +++ ethicml/metrics/accuracy.py @@...

This one would involve flipping some labels

Looking at People Dataset

2

http://chalearnlap.cvc.uab.es/

As used in [this post](https://blog.tensorflow.org/2020/11/applying-mindiff-to-improve-model.html)

add diabetes dataset as used in edwards and storkey

1

Form the censoring representations with an adversary paper "The Diabetes dataset consists of hospital data from the US and the task is to predict whether a patient will be readmitted...

This paper [https://arxiv.org/pdf/1905.12728.pdf](https://arxiv.org/pdf/1905.12728.pdf) landed on arxiv this week. Their main point is that unfair behaviour can occur during data cleaning, in particular how we deal with missing values. They explicitly...