CPM-1-Finetune-Text-Generation icon indicating copy to clipboard operation
CPM-1-Finetune-Text-Generation copied to clipboard

type_error,load_optimizer_states=False好像没起作用怎么回事

Open ZORO-Q opened this issue 2 years ago • 3 comments

Traceback (most recent call last): File "finetune_text_generation.py", line 324, in main() File "finetune_text_generation.py", line 208, in main model, optimizer, lr_scheduler = setup_model_and_optimizer(args) File "/CPM/utils.py", line 510, in setup_model_and_optimizer args.iteration = load_checkpoint(model, optimizer, lr_scheduler, args) File "/CPM/utils.py", line 281, in load_checkpoint checkpoint_name, sd = model.load_checkpoint(args.load, iteration, load_module_strict=False, load_optimizer_states=False, load_lr_scheduler_states=False) File "/usr/local/lib/python3.6/dist-packages/deepspeed/runtime/engine.py", line 1196, in load_checkpoint load_lr_scheduler_states=load_lr_scheduler_states) File "/usr/local/lib/python3.6/dist-packages/deepspeed/runtime/engine.py", line 1231, in _load_checkpoint self.optimizer.load_state_dict(checkpoint['optimizer']) File "/usr/local/lib/python3.6/dist-packages/torch/optim/optimizer.py", line 108, in load_state_dict saved_groups = state_dict['param_groups'] TypeError: 'NoneType' object is not subscriptable

ZORO-Q avatar Aug 05 '22 09:08 ZORO-Q

昨天还能跑起来,今天就一直出了这个问题

ZORO-Q avatar Aug 05 '22 09:08 ZORO-Q

deepspeed换个版本

zhenhao-huang avatar Aug 06 '22 08:08 zhenhao-huang

deepspeed换个版本

好的,我试试

ZORO-Q avatar Aug 07 '22 07:08 ZORO-Q