Fair MapieQuantileRegressor

[gmartinonQM]

Hi all, I recently came accross a paper leveraging prediction intervals to improve fairness of algorithm. The idea is that, given conformal inference often provides marginal coverage, it could happen that there is a marginal coverage of 90% globally, but a 60% coverage for say women, and 100% coverage for say men. The equalized coverage approach inflates prediction interval so as to get a conditional coverage of 90% in the sense that both men and women have a coverage of 90%. Thus if a sensitive groups is more difficult to predict, this would readily appear in the prediction interval width.

Technically, this is very simple :

• Method 1 : train a model on the whole training set, and calibrate separately between groups
• Method 2 : train and calibrate a model for each group

The paper applies these techniques to Quantile regression with mathematical guarantees.

I think this would be a good direction to push MAPIE further and draw connections with the broader field of Trustworthy AI.

Source : https://arxiv.org/pdf/1908.05428.pdf

[nmdwolf]

I agree that adding so-called Mondrian Conformal Prediction to MAPIE would make for an excellent enhancement. Much research and applications these days include conditioning in one form or another, so this would definitely boost applicability.

[LacombeLouis]

Hi @gmartinonQM, thank you for your issue. Indeed, this would be a good option to have a good conditional coverage. We plan to tackle this by implementing a conditional method by the end of the year!