Tom

icon location Hong Kong icon location I am Tom, currently year 2 BEng(DS&E) in HKU.

Results 4 comments of Tom

Yi 1.5 Model Support

I think it is possible to fine tune Yi-1.5, just replace the model name to be `01-ai/Yi-1.5-9B` or `unsloth/Yi-1.5-6B-bnb-4bit` or other models. I used unsloth to fine tune `01-ai/Yi-1.5-9B`, here...

Question - Do you still need enough RAM to run the models?

I allocated 20 GB of ram + 20 GB of swap file for my WSL2 (I run on WSL2).

Very high memory usage for a small model

Just notice that `n_gpu` is set to be 1 on FSDP and Deepspeed. Now I'm even more confused... ![image](https://github.com/user-attachments/assets/4e8fcf5a-584b-4ba2-9c25-9851aca61a48)

Using CPU when resume training from checkpoint @ patch 2024.7

Yeah