QCNet icon indicating copy to clipboard operation
QCNet copied to clipboard

How to load checkpoint and continue training?

Open CoconutSweet999 opened this issue 10 months ago • 1 comments

I wonder how to load checkpoint and continue training. Because of the OOM (maybe at 30 epochs), I have to load checkpoint and continue training after OOM. I try to add ckpt_path = '....ckpt' (path from lightning_logs/checkpoint), but it seems useless. How should I solve this problem? Thanks a lot!

CoconutSweet999 avatar Apr 26 '24 01:04 CoconutSweet999

For example, you can have a try to use different lr or use the default lr.

model = QCNet.load_from_checkpoint(checkpoint_path=ckpt_path)
model = QCNet.load_from_checkpoint(checkpoint_path=ckpt_path, lr=lr)

SunHaoOne avatar Apr 26 '24 05:04 SunHaoOne