ml-explore
How to do the LoRA fine-tuning incrementally
Firstly I finished the LoRA fine-tuning with some training datasets, the result is good. Later I hope to add more training datasets, I have tried the following two methods, both of them cannot get the results as expected. Please shed some light on it.
Thanks.
The first LoRA Training:
python -m mlx_lm.lora
--model ~/models/Qwen1.5-7B-Chat
--train
--data ~/data
--adapter-file ./first_adapters.npz
--iters 400
Incremental method1 :
python -m mlx_lm.lora
--model ~/models/Qwen1.5-7B-Chat
--resume-adapter-file ./first_adapters.npz
--train
--data ~/newdata
--iters 300
Incremental method 2:
2.1 Fused the first_adapters.npz and the base model.
python -m mlx_lm.fuse
--model ~/models/Qwen1.5-7B-Chat
--adapter-file ./first_adapters.npz
--save-path ./fused
the mlx_generate result using the fused model is also good.
2.2 Did the LoRA training with the new data set.
python -m mlx_lm.lora
--model ~/fused
--train
--data ~/newdata
--iters 300
both of them cannot get the results as expected
Either approach is quite reasonable and should work. What happens when you try?
both of them cannot get the results as expected
Either approach is quite reasonable and should work. What happens when you try?
I suspect it is due to the contents of training datasets, should I need to involve some datas from the previous training datasets into the new one? I just did some evaluation for incremental LoRA fine-tuning, so I put the same contents from train.jsonl into the valid.jsonl for the new training datasets, Is it right?
both of them cannot get the results as expected
Either approach is quite reasonable and should work. What happens when you try?
For example I use with fused only for Q "who are the members of *** team?", Answer is perfect. After adding some new training datasets, I use the fused model with the new adapters.npz, Q "who are the members of *** team?", the answer contains some people names from the new training sets, but these names are from totally different questions of new datasets.
If I combine the old and new datasets into only a train.jsonl and valid.jsonl, do the LoRA training from the scratch , the result is as expected.
I use Chinese characters in training datasets.
@southkorea2013 did you ever try mixing old data with new data? I think that should probably solve your issue. I'm closing this as it's not really a bug / specific request. If you want to start a discussion about please go ahead.