if001

Bloom does not have q_proj and v_proj. If you specify query_key_value for target_modules, it will work. https://github.com/huggingface/peft/blob/main/src/peft/mapping.py#L46 However, I am not sure why key_layer should be included for Bloom. I...