torch-rnn
torch-rnn copied to clipboard
Option to store checkpoint on Ctrl+C interrupt
Sometimes, I need to interrupt training. And if you have a large model and checkpoint every 1000 iterations, as set by default, then you could lose a lot of progress, especially if you're halfway to the next checkpoint. Option to store checkpoint on interrupt would be useful for that case.
I am thinking about the same problem. However, you can set up an interrupt handler and call your own saving function inside the handler. (It's easy to do it in C, but I didn't find the equivalent function in Lua)