trainable params: 0

[doxtor6]

When doing finetuning, Lora-weight is loaded with peft without an error but has trainable params: 0.

[MurphyJUAN]

I have the same issue :(

[MurphyJUAN]

I think you need to add is_trainable=True when you use PeftModel: model = PeftModel.from_pretrained(model, './run_alpaca_lora/alpaca-lora-ckpt', torch_dtype=torch.float16, device_map={'': 0}, is_trainable=True )

When doing finetuning, Lora-weight is loaded with peft without an error but has trainable params: 0.

[peppa-xyz]

@Mikubill also encountered this problem, and also set is_trainable=True, but it still shows trainable params: 0

[freQuensy23-coder]

Same problem, LoRA adapters does has no grad, inspite of is_trainable=True,

from peft import prepare_model_for_kbit_training, LoraConfig, get_peft_model
from transformers import LlamaForCausalLM, LlamaTokenizer
import torch

load_in_8bit = True
model = LlamaForCausalLM.from_pretrained(
    base_model,
    load_in_8bit=load_in_8bit,
    torch_dtype=torch.float16,
    device_map=device_map,
)   

tokenizer.pad_token_id = 0
tokenizer.padding_side = "left"  

if load_in_8bit:
    model = prepare_model_for_kbit_training(model)


config = LoraConfig.from_pretrained('path',
                                    is_trainable=True, 
                                    torch_dtype=torch.float16,  device_map={'': 0}
                                    )

model = get_peft_model(model, config)

#%%
model.print_trainable_parameters() # trainable params: 0 || all params: 6,742,609,920 || trainable%: 0.0

[freQuensy23-coder]

config = LoraConfig.from_pretrained('path', is_trainable=True, torch_dtype=torch.float16, device_map={'': 0} ) config.inference_mode = False

this helps me

[ivsanro1]

prepare_model_for_kbit_training freezes all the layers of the model so it makes sense they're not trainable after the line:

model = prepare_model_for_kbit_training(model)