The_Stallion comments

Results 4 comments of


                                            The_Stallion

ValueError: Target modules [q_proj,v_proj] not found in the base model

I tried with a different model and it works fine `from peft import LoraConfig, TaskType peft_config = LoraConfig(task_type=TaskType.SEQ_2_SEQ_LM, inference_mode=False, r=8, lora_alpha=32, lora_dropout=0.1) from transformers import AutoModelForSeq2SeqLM model = AutoModelForSeq2SeqLM.from_pretrained("bigscience/mt0-large") from...

fsdp+qlora mixtral 8x22B: RuntimeError: Only Tensors of floating point and complex dtype can require gradients

I am using qlora with 4-bit quant but somehow i have the same error. For more detail, this is the config i used : `BitsAndBytesConfig { "_load_in_4bit": true, "_load_in_8bit": false,...

Fail to pip install networkx-metis

Hello, `pip install networkx-metis` and https://pypi.org/project/networkx-metis/ doesn't work for me neither. Plus i tried to clone the repo and build -> install and i got compilation errors with Cython. (I...

RuntimeError: only Tensors of floating point dtype can require gradients for QLoRA since transformers 4.40

> Hi @dipanjanS ! Thanks for the issue, I had a deeper look. Previously there was a silent bug in transformers that was quantizing the `pre_classifier` layer, which shouldn't happen...