does rouge_score = evaluate.load("rouge") support other languages?

[lixnvege]

Hi all, I am doing summarization tasks using rouge score which I found in https://huggingface.co/learn/nlp-course/chapter7/5?fw=pt :

!pip install rouge_score

import evaluate

rouge_score = evaluate.load("rouge")
scores = rouge_score.compute(
    predictions=[generated_summary], references=[reference_summary]
)

But the scores for some languages are even 0 like Marathi or Bengali. Does it support languages other than English? if not, is there any method that I could use?

Thanks!

[Whisht]

I have the same question for Chinese evaluation too #465 . But after scanning the documents of rouge-score, I am afraid that I have to turn to another implementation for Chinese. As there are many packages for Chinese, I suggest you find another package for the computation of Marathi or Bengali. And then just replace rouge_score.compute() with your implementation.