AttributeError: 'ChatGLMForConditionalGeneration' object has no attribute 'init_lora_weight'

[yigeliangzai]

模型能读取到，但不能初始化ChatGLM模型中一些LORA (Low-Rank Adaptation)适配器的权重。请问该怎么解决

[merlintang]

does it fixed. @mikecovlee

[mikecovlee]

Thank you for choosing mLoRA! Currently, mLoRA only supports LLaMA and its variant models. We are working on expanding support to other models, including GLM, in future updates.

In the meantime, you can check out our MoE-PEFT, which is a variant of mLoRA focused on optimizing Mixture of Experts (MoE). It offers similar capabilities for multi-LoRA configurations and could be a suitable alternative for your needs.