ai-notebooks icon indicating copy to clipboard operation
ai-notebooks copied to clipboard

Published 20 hours ago • verified publisher icon geohot

Loss was off by factor 10

Open Popeyef5 opened this issue 4 years ago • 0 comments

The loss in the MNIST from scratch notebook is off by a factor of 10 due to the averaging. The averaging in NLLLoss is done over the batch, like in https://pytorch.org/docs/stable/generated/torch.nn.NLLLoss.html

No meaningful result changes, but nice for completeness.

out variable not deleted because it is used for backprop.

Popeyef5 avatar Feb 06 '21 20:02 Popeyef5