Qwen 0.5B inference 报错

[dongguanting]

Reminder

• [X] I have read the README and searched the existing issues.

Reproduction

template=qwen
path_to_llama_model=/cpfs01/shared/Group-m6/dongguanting.dgt/LLaMA_Factory_sft/checkpoint/Qwen1.5-0.5B_hq_dense_train_all
datasets=("hq_dense_top_2" "hq_dense_top_3_120")

for ((i=0; i<${#datasets[@]}; i++)); do
    path_to_llama_model=${path_to_llama_model[$i]}
    dataset=${datasets[$i]}
    OUTPUT_PATH="/output_result/baseline_44/Qwen1.5-0.5BB_$dataset"

    echo $dataset
    echo $OUTPUT_PATH

    CUDA_VISIBLE_DEVICES=0 python src/train_bash.py \
        --stage sft \
        --model_name_or_path $path_to_llama_model \
        --do_predict \
        --dataset $dataset \
        --template $template \
        --output_dir $OUTPUT_PATH \
        --per_device_eval_batch_size 8 \
        --max_samples 10000 \
        --cutoff_len 2048 \
        --predict_with_generate \
        --fp16 \
        --quantization_bit 4 \
        --max_new_tokens 20
done

Expected behavior

期望预测我的数据

System Info

Others

No response

[codemayq]

感觉是数据集有问题，需要 贴一下数据集样例和 data_info.json 的注册信息

[zhangfan-algo]

也同样会遇到类似的问题

[codemayq]

请有一样问题的同学 贴一下数据集样例和 data_info.json 的注册信息 @zhangfan-algo