pointer_summarizer icon indicating copy to clipboard operation
pointer_summarizer copied to clipboard

Published 20 hours ago • verified publisher icon atulkum

The low rouge scole

Open jiangix01 opened this issue 5 years ago • 3 comments

Hello, Thanks for your valuable code. I have a question. When I run the code for 500k iterations. I get the rouge result: ROUGE-1: rouge_1_f_score: 0.2648 with confidence interval (0.2574, 0.2725) rouge_1_recall: 0.3672 with confidence interval (0.3572, 0.3771) rouge_1_precision: 0.2162 with confidence interval (0.2095, 0.2229)

ROUGE-2: rouge_2_f_score: 0.0944 with confidence interval (0.0881, 0.1005) rouge_2_recall: 0.1315 with confidence interval (0.1227, 0.1396) rouge_2_precision: 0.0772 with confidence interval (0.0717, 0.0824)

ROUGE-l: rouge_l_f_score: 0.2348 with confidence interval (0.2277, 0.2419) rouge_l_recall: 0.3249 with confidence interval (0.3149, 0.3346) rouge_l_precision: 0.1920 with confidence interval (0.1854, 0.1980)

So, Can you help me ? Thanks a lot.

jiangix01 avatar Mar 05 '19 14:03 jiangix01

did you turn on the coverage loss?

atulkum avatar Mar 06 '19 00:03 atulkum

Very thanks for your reply. I do not turn on the coverage loss. However, according to the figure 2 in the readme, the result without coverage loss is also high. My score is very low. So I do not know the reason. Thanks a lot.

jiangix01 avatar Mar 06 '19 08:03 jiangix01

Very thanks for your reply. I do not turn on the coverage loss. However, according to the figure 2 in the readme, the result without coverage loss is also high. My score is very low. So I do not know the reason. Thanks a lot.

Me too. Have you fixed the issue?

Sixy1204 avatar Jun 17 '21 05:06 Sixy1204