Help prompt tuning bloom model

[M-Chimiste]

Hello,

I had a question on why I was getting an error for my LoraConfig. I'm assuming it's just me not understanding something. When I put target modules for ["q_proj", "v_proj"] with bloom/z I get an error (ValueError: Target modules ['q_proj', 'v_proj'] not found in the base model. Please check the target modules and try again.)

Full Config:

    r=LORA_R,
    lora_alpha=LORA_ALPHA,
    task_type=TaskType.CAUSAL_LM,
    target_modules=["q_proj", "v_proj"],  # <- This will cause an error with bloom and I don't know why
    lora_dropout=LORA_DROPOUT,
    bias="none"
)

Any help better understanding this is appreciated. When I look at the Models Matrix Support it looks like this should work, since I'm able to do this with LLaMA.

Thanks for any information. If there is a need for more information on the actual training code I can attach.

[if001]

Bloom does not have q_proj and v_proj. If you specify query_key_value for target_modules, it will work. https://github.com/huggingface/peft/blob/main/src/peft/mapping.py#L46

However, I am not sure why key_layer should be included for Bloom. I would like to get more details on this as well.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.