unitxt icon indicating copy to clipboard operation
unitxt copied to clipboard

Support disambiguation of scores from metrics that returns same score names

Open yoavkatz opened this issue 1 year ago • 2 comments

Today, If metrics, returns the same score (e.g. "f1") then the first metric score is returned, and the second metric is ignored.

metrics ["metrics.bert_score.distilbert_base_uncased","metrics.bert_score.deberta_base_mnli"],

It would be good to be able to add a prefix:

metrics ["metrics.bert_score.distilbert_base_uncased[score_prefix="distilbert_]","metrics.bert_score.deberta_base_mnli[score_prefix="deberta_]"],

The score prefix will be appended to any score returned by the metrics (except score_name, and score which are reserved names).

yoavkatz avatar May 06 '24 07:05 yoavkatz

I think @arielge also dealt with it. @arielge if you have more to add to this issue feel free.

elronbandel avatar May 06 '24 07:05 elronbandel

Indeed, I encountered this issue in the context of overwrite args - https://github.com/IBM/unitxt/issues/737, but nothing to add otherwise

arielge avatar May 06 '24 07:05 arielge