nn-zero-to-hero
nn-zero-to-hero copied to clipboard
stepi vs lossi plot bug fixed in makemore2 mlp notebook
To keep track of training steps and loss you initialised lossi and stepi lists. Then while running the training loop multiple times loss was appended to lossi correctly, but you were appending i to stepi and that restarting from 0 again. Hence when you ran the training loop multiple times in the video, the loss was appended but with duplicate steps.
#################################### To keep track of actual steps I did: step = 0 # initialise
for i in range(50000): step += 1 stepi.append(step)
instead of: for i in range(50000): stepi.append(i)
#####################################
I have uploaded the plots with incorrect and correct step tracking. I ran the training loop 2 times in both cases with 50000 training steps each time. But one plot is only showing 50000 steps instead of 100000 steps.
These are the plots I am talking about. Both have been run for 100000 epochs
Sorry for a messy pull request I am still learning.