LLaMA-Factory

LLaMA-Factory copied to clipboard

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction 运行mmlu评估时，results.json中的结果少了一项。 results.json中没课结果只有四个答案，如下： ```json "abstract_algebra": { "0": "B", "1": "C", "2": "A", "3": "A" },...

BGFGB

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction 量化参数  ### Expected behavior _No response_ ### System Info 使用版本0.7.0 ### Others Meta-Llama-3-8B-Instruct进行微调，合并权重后的模型正常，推理没有异常。对模型进行int4量化后，量化模型推理出现问题。...

litterGuy

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction #!/bin/bash CUDA_VISIBLE_DEVICES=0 python ../../src/train_bash.py \ --stage dpo \ --do_train \ --model_name_or_path /mnt/data/legalexp/LLM_exp/MiniCPM/MiniCPM-2B-sft-bf16 \ --adapter_name_or_path...

hecongqing

KTO和DPO显存占用对比

3

相同配置下，实验下来KTO比DPO更省显存 请问这是为什么？

FangLi1

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction I pulled new code and ran Accelerate +FSDP + Qlora training, but encountered an...

tanghui315

使用KTO进行多机训练过程中再进行验证，报错RuntimeError: still have inglight params [{id：388, "status":"AVALIBLE"}]

5

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction 最新的版本llama-factory，使用KTO进行多机训练过程中再进行验证，报错RuntimeError: still have inglight params [{id：388, "status":"AVALIBLE"}] ### Expected behavior _No response_ ### System...

onex7777

llama3-8B-base模型全量微调mmlu掉点

6

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction ### 训练配置文件 1 # model 2 model_name_or_path: ../models/Meta-Llama-3-8B/ 3 4 # method 5 stage:...

sweetning0809

请问现在支持流水线并行或者张量并行等模型并行的方式吗？不是数据并行

5

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction 目前运行的脚步貌似都是数据并行。计划两张24G的卡Lora微调Qwen-14B，但单张卡放不下，又不想量化降低性能。因此询问一下该仓库是否支持模型并行。 ### Expected behavior _No response_ ### System Info _No response_ ### Others _No...

jqwangai

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction The code will pre-tokenize all samples before loading models. However, when the dataset is...

XpastaX

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction 1.在4090×2上训练的checkpoint，per_device_train_batch_size=2，gradient_accumulation_steps=8，换到A100 40G×4机器上，per_device_train_batch_size=8，gradient_accumulation_steps=8，每个epoch的step由原来的13000多缩小到了6000多，这是为什么，也不是线性关系啊，应该缩小到1/8才对。 2.A100的机器上的adapter模型再加载到4090上训练，报错 TypeError: TrainerState.__init__() got an unexpected keyword argument 'stateful_callbacks'。 3.我在做SFT的lora微调，用的int4量化，还有什么办法能加速吗，4*A100 40G，per_device_train_batch_size=8，每张卡大约占用20多G，设置到10就可能OOM。 谢谢大佬回复。...

sunfan1997