instruct-eval
instruct-eval copied to clipboard
AutoModelForCausalLM supports llama models now
In newer versions of the transformer library, AutoModelForCausalLM can properly identify llama models.
There's therefore no need anymore for the LlamaModel class. Llama models run with --model_name causal.
The only hiccup I experienced was an error about the generate function receiving token_type_ids. I fixed this by adding to my tokenizer_config.json the lines
"model_input_names": [
"input_ids",
"attention_mask"
],
This could be addressed within flan-eval by setting return_token_type_ids=False in CausalModel's call to the tokenizer
self.tokenizer(prompt, return_tensors="pt", return_token_type_ids=False)
Thanks! Could you please do a PR?