Minimize how often we load args.resume

Open achalddave opened this issue 2 years ago • 1 comments

Currently, we load args.resume potentially up to 3 times. This can be pretty slow for big models, and we should avoid re-loading it in these spots: https://github.com/mlfoundations/open_lm/blob/97d0a4ac592e300c7a1688c4a48584b027b45af8/open_lm/main.py#L110-L156

Nov 06 '23 05:11 achalddave

In the current iteration, I believe our main offender is: https://github.com/mlfoundations/open_lm/blob/main/open_lm/main.py#L142-L151.

Dec 11 '23 00:12 achalddave