swift icon indicating copy to clipboard operation
swift copied to clipboard

ms-swift: Use PEFT or Full-parameter to finetune 250+ LLMs or 25+ MLLMs

Results 206 swift issues
Sort by recently updated
recently updated
newest added

https://huggingface.co/OpenGVLab/InternVL-Chat-V1-5 希望可以支持一下微调,实际使用下来效果确实很好

more models

**Describe the feature** Please consider adding support for ORPO training. It extends preference optimization into sft stage and shows promising results. **Paste any useful information** [Paper](https://arxiv.org/abs/2403.07691) and [Support from Huggingface...

多模态 minicpmv2的训练,使用的尺寸是多大,原始minicpm v-2的训练尺寸可以很大,查看源码只看到minicpmv2的输入尺寸是448,训练和推理时对于大图会在原图上进行处理吗,还是会统一直接resize缩放到448?

# PR type - [ ] Bug Fix - [x] New Feature - [ ] Document Updates - [ ] More Models or Datasets Support # PR information Refactor the...

# PR type - [ ] Bug Fix - [x] New Feature - [ ] Document Updates - [ ] More Model or Dataset Support # PR information Previous PR:...

**Describe the bug** yi-6B-chat可训练参数在`lisa_activated_layers设置1/2/4都显示`[INFO:swift] LlamaForCausalLM: 6061.0355M Params (524.2921M Trainable [8.6502%]), 33.5565M Buffers.`,这个计算是不是有问题?

bug

请问在微调中我们使用的数据集有29450条,batch_size=1, train_dataset_sample=-1, 为什么训练一个epoch使用约1800steps? 按我的理解batch_size=1每次加载一条数据,step应该与数据数量一致? 感谢您的回答!

question