lecun1989-repro icon indicating copy to clipboard operation
lecun1989-repro copied to clipboard

Update eval loss in modern.py.

Open chenyuxyz opened this issue 2 years ago • 1 comments

I would like to update the loss numbers in the comment section too, but I could not reproduce your numbers.

For repro.py, I am getting 23 eval: split train. loss 5.344314e-03. error 0.86%. misses: 63 eval: split test . loss 2.703927e-02. error 3.54%. misses: 71

And for modern.py, I am getting 80 eval: split train. loss 4.209032e-02. error 1.11%. misses: 80 eval: split test . loss 5.929652e-02. error 1.99%. misses: 39

Not clear if this is due to different data subsets or library behavior. I am using numpy==1.20.3, torch==1.11.0 and torchvision==0.12.0.

chenyuxyz avatar Mar 16 '22 14:03 chenyuxyz

@leastcompressible I think the typo you discovered is responsible for the train and test losses being identical in the documentation for modern.py

https://github.com/karpathy/lecun1989-repro/blob/8553f52c8d0a51a4bbbabf98e005cef28a22a36b/modern.py#L11-L40

as well as in the accompanying blogpost (snippet 1, snippet 2)

vtjeng avatar Mar 20 '22 22:03 vtjeng