Call `model.eval()` before generating?

[gustavdelius]

I understand why we have to call model.eval() before calculating the average loss in estimate_loss(). But should we not similarly call model.eval() before we start generating from the model?

[manudwd]

You should be calling model.eval() at the time of inference, that is, if you want to set the affect of weights post normalization and dropout. Without setting model.eval() there is no effect of dropout and normalization at the time of testing. I believe that is just something that was forgotten.