Scheduling Teacher Forcing Ratio as Curriculum Learning

[kylegao91]

I sometimes notice that not using teacher forcing at all gives better results at inference time than using teacher forcing all the time. This paper provides evidence for this behavior and proposed scheduled sampling as a curriculum learning approach for training seq2seq.