Dhruv Jain comments

Results 5 comments of


                                            Dhruv Jain

does xlora train not support llama2?

@crossxxd can you share your training code for mistral 7b base model ? I have been able to put the llama model on the training, however the training is very...

does xlora train not support llama2?

@maohaos2 can you share your code ?

Support for seq2seq models

@yuxiang-guo Yeah I did. But I faced a tensor related error while doing the forward pass. I tried debugging through it but couldn't exactly understand what was the problem here.

Using trl SFTTrainer creates empty adapter.safetensors file while saving when training with LoRA and Deepspeed Zero3

Hii @BenjaminBossan here's the config file: ``` model: # paths llm_path: "google/gemma-3-4b-it" # LoRA lora: True lora_rank: 8 lora_alpha: 16 lora_dropout: 0.05 target_modules: ["q_proj", "v_proj", "up_proj", "down_proj"] max_seq_len: 4096 end_sym:...

Using trl SFTTrainer creates empty adapter.safetensors file while saving when training with LoRA and Deepspeed Zero3

@BenjaminBossan thanks for this, i'll try reproducing this. Will try to find if there is any difference.