Reproducing translation results using the released finetuned checkpoint

[yiqingxyq]

Hi! I evaluated your finetuned checkpoint on java-cs translation but could not get the exactly same results as your paper reported. I got 83.89/64.7 but the paper reported 84.03/65.9. I read that you use beam-search w/o sampling to generate the results, which should not bring randomness, so I'm wondering where did the randomness come from.

This is my output:

I downloaded the checkpoint from here: (and I used translate_java_cs_codet5_base.bin)

Thank you!