nlp-recipes icon indicating copy to clipboard operation
nlp-recipes copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

[FEATURE] save optimizer and amp state into checkpoint

Open daden-ms opened this issue 5 years ago • 0 comments

Description

Currently, in the common.py for transformer models, a checkpoint only saves model state and the optimizer and amp state info is not saved. We can consider saving this info like in https://github.com/NVIDIA/apex#checkpointing

Expected behavior with the suggested feature

Other Comments

daden-ms avatar Feb 18 '20 20:02 daden-ms