55-epoch LR schedule

[soumith]

hey, i see that you have the 55-epoch schedule in your references. With batchnorm, it is much lower, there's a schedule that converges within ~20 epochs.

[andreaskoepf]

Thx for pointing that out, I will try to Google for it. My results on imagenet are currently still worse than I had expected. I will have to do some more debugging and restructuring in order to determine if there is a problem in the impl or if joint training (which is one of the things I do differently from the faster-rcnn paper beside training from scratch instead using a pre-trained net (in order to train multiple different topologies)) is something that hinders a good classification.

[soumith]

Here's a log of a 28-epoch convergence that I have from March: https://gist.github.com/soumith/4766a592cb3645035ef8