Summarization evaluation using `rouge2` is quite slow

[pfliu-nlp]

For example, by running

explainaboard --task summarization --custom_dataset_paths ./data/system_outputs/cnndm/cnndm_mini-dataset.tsv --system_outputs ./data/system_outputs/cnndm/cnndm_mini-bart-output.txt --metrics rouge2

it took more than 15 seconds, which I guess is relatively slower?

[pfliu-nlp]

Hey, @neubig do you think this is normal?

[neubig]

Is this consistent? Or is it slow sometimes and fast sometimes? It could be that the EaaS server is overloaded.