create-million-parameter-llm-from-scratch icon indicating copy to clipboard operation
create-million-parameter-llm-from-scratch copied to clipboard

Published 20 hours ago • verified publisher icon FareedKhan-dev

Getting below error while saving model

Open venkateshmadalainrscr opened this issue 9 months ago • 0 comments

RuntimeError Traceback (most recent call last) Cell In[46], line 6 4 llama_config = GPT2Config.from_dict(MASTER_CONFIG) 5 llama_transformers = GPT2LMHeadModel(config=llama_config) ----> 6 llama_transformers.load_state_dict(llama.state_dict()) 8 # Specify the directory where you want to save the model 9 output_dir = "llama_model_transformers"

File ~/.local/lib/python3.10/site-packages/torch/nn/modules/module.py:2152, in Module.load_state_dict(self, state_dict, strict, assign) 2147 error_msgs.insert( 2148 0, 'Missing key(s) in state_dict: {}. '.format( 2149 ', '.join(f'"{k}"' for k in missing_keys))) 2151 if len(error_msgs) > 0: -> 2152 raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( 2153 self.class.name, "\n\t".join(error_msgs))) 2154 return _IncompatibleKeys(missing_keys, unexpected_keys)

RuntimeError: Error(s) in loading state_dict for GPT2LMHeadModel: Missing key(s) in state_dict:

venkateshmadalainrscr avatar May 12 '24 00:05 venkateshmadalainrscr