pyreft
pyreft copied to clipboard
[P1] ReFT+PEFT by using ReftModel to wrap PeftModel
Descriptions:
By taking a quick look at the PEFT library, it wraps nn.module as a PEFT nn.module which accepts gradient, is trainable, and just like another nn.module. This is highly compatible with ReFT.
We should make ReFT also to support any PEFT model as well. It might work out-of-the-box already. This ticket will track a validation effort. For instance, checking whether trainable parameter prints the correct one when we add LoReFT + LoRA.