LMFlow

LMFlow copied to clipboard

Does it support llama3?

5

Does it support multi-gpu raining llama3 with lisa ?

orderer0001

Causal LM finetuning

3

Hey, Great to see LISA implemented here. As for the background, I am trying to finetune models with LORA other techniques on domain data but the Task i am doing...

harry7171

Memory problem of Lisa finetuning

6

I tried fine-tuning the llama-2-7b model using LoRa on an RTX3090 with 24GB, where the memory usage was only about 17GB. However, when I used the same configuration on an...

lovekdl

About using multiple GPUs to do lisa fine-tuning

4

If there are multiple GPUs, using the lisa method is also a direct script./scripts/run_finetune_with_lisa.sh? Do I need to set multi-GPU parameters?

orderer0001

Hi everyone, I want to use a stable-diffusion model which is good at 256 resolution. If someone could share a finetuned ckpt, that would be great! Thanks a lot!

Pang-0093

Minimum Requirement of GPU for Fine Tuning

4

What is the minimum requirement in order to fine tune small model like openlm-research/open_llama_3b and big model like llama2-7b

vashiegaran

"trust_remote_code=True" problem

1

Hi author, when I use your project to fine-tune a local model, I get the following prompt "trust_remote_code=True" Where should I change the code? Can you provide an example?

yingxin-chen

Running install.sh after git clone requires over 200GB Ram

6

I have a server with 128GB ram and it will be freeze when I follow the quick start procedure. On another server with 512GB ram it's fine. I think adding...

weicheng59

[BUG] LISA: same loss regardless of lisa_activated_layers

17

**Describe the bug** I think there might be something wrong with the current LISA implementation. There is no difference in training loss, no matter how many layers are active. Not...

geronimi73

is support llava model ?

3

can support https://github.com/haotian-liu/LLaVA model ?

awzhgw