llama-recipes
llama-recipes copied to clipboard
meta-llama
evaluation
System Info
python eval.py --model hf --model_args pretrained=/mnt/sdb/zjk/llama2/llama-recipes/Llama-2-7b-hf,dtype="float",peft=/mnt/sdb/zjk/llama2/llama2-lora --tasks hellaswag --num_fewshot 10 --device cuda:0 --batch_size 8 error： 2024-04-02:06:58:12,156 ERROR [eval.py:226] An error occurred during evaluation: module 'lm_eval.tasks' has no attribute 'initialize_tasks'
Information
- [X] The official example scripts
- [ ] My own modified scripts
🐛 Describe the bug
evaluation was changed？ how can i evaluate finetuned model with lora now？
Error logs
2024-04-02:06:58:12,156 ERROR [eval.py:226] An error occurred during evaluation: module 'lm_eval.tasks' has no attribute 'initialize_tasks'
Expected behavior
evaluate finetuned model with lora and the accuracy of specific dataset with cola
Hello! Have you solved this issue? I have the this question as same as yours. If you have solved this question, Please tell me how to do that. Thanks bro.
