neural-question-generation icon indicating copy to clipboard operation
neural-question-generation copied to clipboard

BLEU scores too high in inference progess

Open Shijie97 opened this issue 4 years ago • 2 comments

Hi seanie! Really appreciate for your code share, it does help me deepen my understanding of this paper. But I have some trouble reproducing your code in the inference stage. After inference stage, I moved the two files "generated.txt" and "golden.txt" to the "qgevalcap\result" folder, while there has been a "src.txt" in "qgevalcap\result" folder. I have a few silly questions that may need to be answered.

  1. what on earth is the file "src.txt" for? Does it participate in the calculation of BLEU? Dost it exsit at the beginning?
  2. how is "golden.txt" generated? If this file is invariant, why not saving it as invariant file just like "scr.txt"?
  3. My result for BLEU is too high, it being shown below: Bleu_1: 0.65958 Bleu_2: 0.43423 Bleu_3: 0.30656 Bleu_4: 0.22511 ROUGE_L: 0.49419 CIDEr: 0.56620

I wonder the reason, if you know what the problem is, may you please point it out? Thank you!

Shijie97 avatar Jun 28 '20 18:06 Shijie97

  1. It is dummy file for indexing generated sentences
  2. golden.txt is the ground truth questions.
  3. Did you try my pretrained model?

seanie12 avatar Jul 17 '20 05:07 seanie12

same question, i trained the model at the beginging. the scores is : Bleu_1: 0.51006 Bleu_2: 0.36335 Bleu_3: 0.27556 Bleu_4: 0.21658 METEOR: 0.22948 ROUGE_L: 0.52004 is there something wrong? the scores are too high

leesurelye avatar Mar 18 '22 05:03 leesurelye