got assertion error while loading the CodeT5+ models 16B-Instruct, 6B and 2B like this "AssertionError: Config has to be initialized with encoder and decoder config". how to solve this issue

[Tarak200]

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

checkpoint = "Salesforce/codet5p-2b" device = "cuda" # for GPU usage or "cpu" for CPU usage

tokenizer = AutoTokenizer.from_pretrained(checkpoint) model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint, torch_dtype=torch.float16, trust_remote_code=True).to(device)

encoding = tokenizer("def print_hello_world():", return_tensors="pt").to(device) encoding['decoder_input_ids'] = encoding['input_ids'].clone() outputs = model.generate(**encoding, max_length=15) print(tokenizer.decode(outputs[0], skip_special_tokens=True))

used the same code from huggingface, encountered the following error : AssertionError: Config has to be initialized with encoder and decoder config, How to resolve this issue?

[Roisan]

Hi @Tarak200 , I am also getting the same error while inferencing using Codet5p 2B model. Were you able to find a solution? Thanks.

[angelocurti]

Hi @Roisan @Tarak200 , I am getting the same error. Did you find a solution? Thanks!

[Roisan]

@angelocurti Hello, I haven't been able to find a solution. Any suggestions from your end will be highly appreciated.

Thank you.

[ardhiwiratamaby]

Creating the generation config solved my problem.

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer from transformers import GenerationConfig import torch

checkpoint = "Salesforce/codet5p-2b" device = "cuda" # for GPU usage or "cpu" for CPU usage

Select a device (CPU if no GPU is available, otherwise the first GPU)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu") print(f"Selected device: {device}")

tokenizer = AutoTokenizer.from_pretrained(checkpoint) model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint, torch_dtype=torch.float16, trust_remote_code=True).to(device)

python_function = "write code for matrix multiplication" encoding = tokenizer(python_function, return_tensors="pt").to(device) encoding['decoder_input_ids'] = encoding['input_ids'].clone()

# Create a GenerationConfig with custom settings generation_config = GenerationConfig( max_length=950 )

outputs = model.generate(**encoding, generation_config=generation_config) print(tokenizer.decode(outputs[0], skip_special_tokens=True))

[Roisan]

@ardhiwiratamaby Thank you for providing the solution, I am now able to infer the 2B model.

[latuan1]

Hello, @ardhiwiratamaby, I got assertion error while full fine-tuning CodeT5+ 2B: File "/root/anaconda3/envs/train-model/lib/python3.11/site-packages/transformers/trainer.py", line 2245, in train return inner_training_loop( ^^^^^^^^^^^^^^^^^^^^ File "/root/anaconda3/envs/train-model/lib/python3.11/site-packages/transformers/trainer.py", line 2627, in _inner_training_loop self._maybe_log_save_evaluate( File "/root/anaconda3/envs/train-model/lib/python3.11/site-packages/transformers/trainer.py", line 3103, in _maybe_log_save_evaluate self._save_checkpoint(model, trial) File "/root/anaconda3/envs/train-model/lib/python3.11/site-packages/transformers/trainer.py", line 3200, in _save_checkpoint self.save_model(output_dir, _internal_call=True) File "/root/anaconda3/envs/train-model/lib/python3.11/site-packages/transformers/trainer.py", line 3902, in save_model self._save(output_dir) File "/root/anaconda3/envs/train-model/lib/python3.11/site-packages/transformers/trainer.py", line 4006, in _save self.model.save_pretrained( File "/root/anaconda3/envs/train-model/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3337, in save_pretrained misplaced_generation_parameters = model_to_save.config._get_non_default_generation_parameters() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/anaconda3/envs/train-model/lib/python3.11/site-packages/transformers/configuration_utils.py", line 1079, in _get_non_default_generation_parameters default_config = self.class() ^^^^^^^^^^^^^^^^ File "/root/.cache/huggingface/modules/transformers_modules/Salesforce/codet5p-2b/0083d4d638746e6c9ee3dbd504e6dd68738e3c87/configuration_codet5p.py", line 78, in init "encoder" in kwargs and "decoder" in kwargs AssertionError: Config has to be initialized with encoder and decoder config. How to solve this issue.