ctrl benchmarking with GPT-2

benchmarking with GPT-2

Open leejason opened this issue 6 years ago • 2 comments

Any suggestion for benchmarking CTRL with GPT-2? Say, loss value, PPL, or any metric to measure text generation quality?

Oct 19 '19 09:10 leejason

Not a direct answer to your question, but this (timely) article by @chiphuyen is really good

https://thegradient.pub/understanding-evaluation-metrics-for-language-models/

Oct 22 '19 20:10 julien-c

very helpful & thanks

Oct 23 '19 12:10 leejason