torcheval icon indicating copy to clipboard operation
torcheval copied to clipboard

The score computed by `multiclass_f1_score` for binary classification is wrong. It is not f1 score but accuracy.

Open LittletreeZou opened this issue 1 year ago • 0 comments

🐛 Describe the bug

The score computed by multiclass_f1_score for binary classification is wrong. It is not f1 score but accuracy, as shown in following code:

import torch
from torcheval.metrics.functional import multiclass_f1_score, binary_f1_score

actual = torch.repeat_interleave(torch.tensor([1, 0]), repeats=torch.tensor([100, 100]))
pred = torch.repeat_interleave(torch.tensor([1, 0, 1, 0]), repeats=torch.tensor([55, 45, 34, 66]))

multiclass_f1_score(pred, actual, num_classes=2)
# tensor(0.6050)

(actual == pred).sum()/200
# tensor(0.6050)

binary_f1_score(pred, actual)
# tensor(0.5820)

Versions

torcheval 0.0.7

LittletreeZou avatar Jan 18 '24 13:01 LittletreeZou