lm-evaluation-harness icon indicating copy to clipboard operation
lm-evaluation-harness copied to clipboard

Regarding metric chrf's implementation

Open PinzhenChen opened this issue 6 months ago • 1 comments

Hi,

In this repo, the chrf metric implementation calls sacrebleu.corpus_chrf() with default parameters: character order 6 and word order 0. Perhaps in metric.py it would be nice to include those two optional parameters so users can specify this in task.yml, for example to facilitate chrf++ which uses word order 2?

Thanks!

PinzhenChen avatar Aug 28 '24 10:08 PinzhenChen