LoRA icon indicating copy to clipboard operation
LoRA copied to clipboard

Clarifying questions about the paper

Open kilimanj4r0 opened this issue 1 year ago • 0 comments

Hi, Thank you for sharing the source code. I really enjoy the work you propose.

While reading the paper and reproducing the results I got a couple of questions:

  1. In the Table 3 the row with the results for GPT2-M AdapterH written without asterix. But I couldn't find any source code that implements GPT2-M with AdapterH. Is this a typo?
  2. Regarding the computation of METEOR for WebNLG and DART datasets. I can't reproduce the result of this metric with the script you proposed from GenerationEval repo. I wrote my own script that evaluates WebNLG and DART using HuggingFace evaluate library and got same (very close) results. So, how did you obtain such METEOR score?

kilimanj4r0 avatar Nov 16 '22 08:11 kilimanj4r0