torch-rnn icon indicating copy to clipboard operation
torch-rnn copied to clipboard

Option to store checkpoint on Ctrl+C interrupt

Open ostrosablin opened this issue 8 years ago • 1 comments

Sometimes, I need to interrupt training. And if you have a large model and checkpoint every 1000 iterations, as set by default, then you could lose a lot of progress, especially if you're halfway to the next checkpoint. Option to store checkpoint on interrupt would be useful for that case.

ostrosablin avatar Jun 20 '16 10:06 ostrosablin

I am thinking about the same problem. However, you can set up an interrupt handler and call your own saving function inside the handler. (It's easy to do it in C, but I didn't find the equivalent function in Lua)

sudongqi avatar Oct 04 '16 22:10 sudongqi