Fengshenbang-LM
Fengshenbang-LM copied to clipboard
KeyError: 'global_step'
Traceback (most recent call last):
File "finetune.py", line 194, in <module>
trainer.fit(model, datamoule, ckpt_path=args.load_ckpt_path)
File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 696, in fit
self._call_and_handle_interrupt(
File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 650, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 735, in _fit_impl
results = self._run(model, ckpt_path=self.ckpt_path)
File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1110, in _run
self._restore_modules_and_callbacks(ckpt_path)
File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1065, in _restore_modules_and_callbacks
self._checkpoint_connector.restore_model()
File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/checkpoint_connector.py", line 174, in restore_model
self.trainer._call_lightning_module_hook("on_load_checkpoint", self._loaded_checkpoint)
File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1550, in _call_lightning_module_hook
output = fn(*args, **kwargs)
File "finetune.py", line 163, in on_load_checkpoint
global_step_offset = checkpoint["global_step"]
KeyError: 'global_step'
Could you please give more background information ? e.g., version of pytorch lightning and which model is loaded?