dart icon indicating copy to clipboard operation
dart copied to clipboard

Update Evaluation used for V.1.1.1

Open jordiclive opened this issue 3 years ago • 2 comments

The references in /evaluation/dart_reference are not for the current version. Can you replace with the new references and share the tokenization script that is done to the predictions.

I am getting very different BLEU scores depending on tokenization, and how many references I use. As there are up to ~30 for a few examples.

I would like to directly compare against the README leaderboard.

jordiclive avatar Jul 24 '21 12:07 jordiclive

Upvoting this, since having the same issue here.

lxuechen avatar Aug 01 '21 00:08 lxuechen

How to run BART in the model. Could you provide more details about running environment and python script.

LemonQC avatar Aug 18 '21 07:08 LemonQC