codingma

recommanded gradio verision is 4.21.0， please install this version and try again. bless

sorry, not supported now

Please first check if the original model can infer properly before training. Then check the log and GPU status when do inference process by the way. My System is "win11,...

> Please first check if the original model can infer properly before training. Then check the log and GPU status when do inference process by the way. My System is...

full 的时候，会忽略lora_target 参数

会的，cutoff 会导致样本的长度和原始的不一致，导致template填充的结果里信息丢失

使用非 Mixtral 类模型会有问题吗？ 需要看一下是不是 MOE的模型类型不同导致的。

目前看到 https://github.com/hiyouga/LLaMA-Factory/issues/2933 跟你是类似的问题，我们有空会进一步排查。

使用 fewshot 的 template 试一下，我的结果是 Average: 61.67 STEM: 48.18 Social Sciences: 74.15 Humanities: 55.64 Other: 70.67

python ./src/evaluate.py \ --model_name_or_path /media/codingma/LLM/llama3/Meta-Llama-3-8B \ --template fewshot \ --task mmlu \ --split validation \ --lang en \ --n_shot 5 \ --batch_size 4