AtacWorks icon indicating copy to clipboard operation
AtacWorks copied to clipboard

Fix resume functionality in training.

Open ntadimeti opened this issue 5 years ago • 0 comments

Currently resume only reads the weights from an existing model and starts training with them instead of randomly initialized weights. This is only partially correct. Ideally, we would save all the states of training in a checkpoint file and resume training exactly where we left off (including lr, random seed, same batch etc)

ntadimeti avatar Feb 27 '20 18:02 ntadimeti