lovekdl

icon location Hi, I am an undergraduate student from Beijing Jiaotong University, and I am entering Peking University in 2025. I'm interested in Optimization in LLM Training.

Results 1 issues of lovekdl

Memory problem of Lisa finetuning

5 comment

I tried fine-tuning the llama-2-7b model using LoRa on an RTX3090 with 24GB, where the memory usage was only about 17GB. However, when I used the same configuration on an...