peft
peft copied to clipboard
Update OFT to fix merge bugs
changes for BOFT
- fixing small errors in comments of BOFT
- making Conv2D operation consistent with Linear
changes for OFT
- [Important!] fixing the incorrect merging of OFT, it is not an implementation error, but more of an incorrect understanding of how orthogonal fine-tuning works, resulting in a wrong way of merging adapters
- update config.py/layer.py/model.py file to be consistent with other peft methods
- adding additional paramters oft_block_size, oft_dropout, bias
- fixing module dropout implementation error (in orthogonal finetuning, dropout is not the same as LoRA)
- fixing errors in merge and unmerge function in OFT
- making Conv2D operation consistent with Linear