Suchun Xie

Hi @subramen, I'm trying to perturb some dimensions of parameter values in some layers (like 0-5 layers's all parameter) and freeze it, then do a full parameter fintuning. I got...

FSDP Finetuned Model-optimizer and tokenizer

3

Thanks for the tutorials! I have several small questions about the model ft and usage. When doing Full parameter finetune using FSDP only, **Q1: should we use `save_optimizer` to True...

DeepSpeed support for Full Finetuning - FSDP performance is not as good as Deepspeed

3

### 🚀 The feature, motivation and pitch I trained the current code with FSDP to full fine-tune Llama2, it is very quick, but it turns out the performance is even...

您好，请问如何分别传入train_datasets.json 和eval_datasets.json， 看了readme好像没有发现如何更改。谢谢 还有一个是，看源代码里面只有llama3的Finetune代码，没有llama2的，请问special token啥的会有影响吗，尤其是推理的时候需要保持的格式