instruct-eval icon indicating copy to clipboard operation
instruct-eval copied to clipboard

AutoModelForCausalLM supports llama models now

Open passaglia opened this issue 2 years ago • 1 comments

In newer versions of the transformer library, AutoModelForCausalLM can properly identify llama models.

There's therefore no need anymore for the LlamaModel class. Llama models run with --model_name causal.

The only hiccup I experienced was an error about the generate function receiving token_type_ids. I fixed this by adding to my tokenizer_config.json the lines

"model_input_names": [
    "input_ids",
    "attention_mask"
  ], 

This could be addressed within flan-eval by setting return_token_type_ids=False in CausalModel's call to the tokenizer

self.tokenizer(prompt, return_tensors="pt", return_token_type_ids=False)

passaglia avatar May 31 '23 08:05 passaglia

Thanks! Could you please do a PR?

soujanyaporia avatar May 31 '23 08:05 soujanyaporia