On Accurate Evaluation of GANs for Language Generation

[howardyclo]

Dear authors:

First, I would like to thank you for the work. It is really helpful for standardizing the development of GAN-based text generation methods.

However, recently Google has just published a paper "On Accurate Evaluation of GANs for Language Generation", arguing that BLEU is not a good, or even a misleading metric for GAN-based text generation methods. (See summary I wrote about this paper). Therefore, In my humble opinion, I think it would be better to report better metric (Reverse LM score and FD, specifically) for those methods that are already implemented on TexyGen.

Best, Howard