nn-zero-to-hero icon indicating copy to clipboard operation
nn-zero-to-hero copied to clipboard

stepi vs lossi plot bug fixed in makemore2 mlp notebook

Open ibadrather opened this issue 2 years ago • 1 comments

To keep track of training steps and loss you initialised lossi and stepi lists. Then while running the training loop multiple times loss was appended to lossi correctly, but you were appending i to stepi and that restarting from 0 again. Hence when you ran the training loop multiple times in the video, the loss was appended but with duplicate steps.

#################################### To keep track of actual steps I did: step = 0 # initialise

for i in range(50000): step += 1 stepi.append(step)

instead of: for i in range(50000): stepi.append(i)

#####################################

I have uploaded the plots with incorrect and correct step tracking. I ran the training loop 2 times in both cases with 50000 training steps each time. But one plot is only showing 50000 steps instead of 100000 steps.

ibadrather avatar Oct 08 '22 09:10 ibadrather

These are the plots I am talking about. Both have been run for 100000 epochs

loss_with_correct_step loss_with_wrong_step

Sorry for a messy pull request I am still learning.

ibadrather avatar Oct 08 '22 09:10 ibadrather