confidenceinterval icon indicating copy to clipboard operation
confidenceinterval copied to clipboard

Confidence Interval for Macro-F1 is always [0,1]

Open vsocrates opened this issue 2 years ago • 2 comments

Hi,

Thanks for a great library! How come the macro-f1 has been hardcoded with a CI of [0,1] in this line?

https://github.com/jacobgil/confidenceinterval/blob/7b47bfdf0ce8ec54a2a2d820b7332fd43b15d0d0/confidenceinterval/metrics.py#L527-L528

I believe we should be using the equation from the paper to compute first variance, then computing the CI from that. Thanks!

vsocrates avatar Jun 07 '23 07:06 vsocrates

Good catch! Will try updating that in the next few days.

jacobgil avatar Jun 07 '23 16:06 jacobgil

Actually that file was an older file and wasn't used by the package. I removed it now. The macro f1 was already implemented here: https://github.com/jacobgil/confidenceinterval/blob/main/confidenceinterval/takahashi_methods.py#L126

jacobgil avatar Jun 09 '23 06:06 jacobgil