squeezeDet icon indicating copy to clipboard operation
squeezeDet copied to clipboard

Published 20 hours ago • verified publisher icon BichenWuUCB

Resume Training

Open FrancescoFornasa opened this issue 6 years ago • 2 comments

Good morning to everyone. I was wondering how should I change the code for allowing the training to be done in repeating iterations. Since i can only train the network during nighttime (due to some intersections on that machine with a colleague of mine), I was thinking on starting the training from a CheckPoint model that i saved. This should be not so hard to implement, but I am struggling in finding a solution by myself, so any suggestion will be very appreciated. Thank you all a lot for this amazing work.

FrancescoFornasa avatar Apr 17 '18 09:04 FrancescoFornasa

I tried out a few solutions, nothing seemed to work. It ended up training from the beginning. Have you made any progress on this front?

aditya1709 avatar Jun 14 '18 20:06 aditya1709

I think if you just point the train_dir flag to the directory of your checkpoints it should load your latest checkpoint file from your previous training job.

Also you may have to comment out lines 338 and 339 in train.py as they will delete your checkpoints in that directory before checking for them. Not sure why those lines are there.

kemusa avatar May 25 '19 11:05 kemusa