pyreft icon indicating copy to clipboard operation
pyreft copied to clipboard

[P1] ReFT+PEFT by using ReftModel to wrap PeftModel

Open frankaging opened this issue 10 months ago • 2 comments

Descriptions:

By taking a quick look at the PEFT library, it wraps nn.module as a PEFT nn.module which accepts gradient, is trainable, and just like another nn.module. This is highly compatible with ReFT.

We should make ReFT also to support any PEFT model as well. It might work out-of-the-box already. This ticket will track a validation effort. For instance, checking whether trainable parameter prints the correct one when we add LoReFT + LoRA.

frankaging avatar Apr 11 '24 09:04 frankaging