Hilde Weerts

Thank you for opening this issue and the thorough and clear description @Harsha-Nori! Let me start by saying I'm super happy that you're picking this up and I think the...

> This is a really important point. On top of documentation on the website, are there any other mechanisms that FairLearn uses to alert users about these types of pitfalls...

> I'm in favor of this being part of MetricFrame. First, because of convenience, but more importantly, because we really need to be able to obtain error bars on difference(),...

I agree that we should not fix the groups in advance - the points raised by @Harsha-Nori are very valid and I think we have the responsibility to avoid possibly...

Thank you for opening this issue @bramreinders97! I personally think this technique would be a good addition to Fairlearn (but I am 100% biased because I would like to get...

100% agree re. naming, @romanlutz!

I agree with @romanlutz! I personally don't worry too much about the proliferation. I think there is something to be said for implementing the Reject Option Classifier from the paper,...

> so you may end up trying lots and lots of combinations until you're finally (somewhat) happy. Sounds like the typical ML process to me!  @koaning

I think you will need `sensitive_features` in `.fit` as well, but other than that this looks like a good place to start from! (Tagging @romanlutz @MiroDudik)

Summary from offline discussion with @bramreinders97: * @bramreinders97 pointed out that including `sensitive_features` in `.fit()` doesn't make a lot of sense here, because `fit` doesn't really do anything apart from...