add reft method

[TranscenderNing]

PR types

New features

PR changes

Add reft in paddlenlp/peft/reft reft ├── pareft │ ├── config.py reft配置文件，继承pavenv.config │ ├── dataset.py reft数据处理，reft方法输入会有一个新的intervention_locations字段，表示干预的token的位置，例如f5+l5表示干预输入的前5个tokne和后5个token │ ├── init.py │ ├── interventions.py 干预网络 │ ├── reft_model.py 创建reft模型，继承pavenv.interventableModel │ ├── reft_trainer.py 重写compute_loss方法，reft方法需要根据配置中的position参数干预对应位置token的隐藏表示 │ └── utils.py 工具类 └── pavenv ├── init.py └── models ├── basic_utils.py 基础的工具类 ├── configuration_intervenable_model.py 创建干预方法的配置 ├── constants.py 常量 ├── init.py ├── intervenable_base.py 这个是方法实现的主要类，在该类中模型添加orward_post_hook，在前向传播过程中hook中提取干预位置的向量，将提取的向量输入干预模型，将干预模型的结果替换对应位置的向量 ├── intervenable_modelcard.py 模型配置的信息 ├── interventions.py 所有干预网络的父类 ├── llama │ └── modelings_intervenable_llama.py llama模型的基础配置 └── modeling_utils.py 模型的一些工具类

Description

[paddle-bot[bot]]

Thanks for your contribution!

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 85.92233% with 58 lines in your changes missing coverage. Please review.

Project coverage is 53.09%. Comparing base (6813e40) to head (0731337). Report is 220 commits behind head on develop.

Files with missing lines	Patch %	Lines
paddlenlp/peft/reft/reft_model.py	78.33%	39 Missing :warning:
paddlenlp/peft/reft/reft_config.py	80.00%	7 Missing :warning:
paddlenlp/peft/reft/modeling_utils.py	94.66%	4 Missing :warning:
paddlenlp/peft/reft/predict.py	92.00%	4 Missing :warning:
paddlenlp/trainer/trainer.py	25.00%	3 Missing :warning:
paddlenlp/peft/reft/interventions.py	98.38%	1 Missing :warning:

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #8819      +/-   ##
===========================================
+ Coverage    52.91%   53.09%   +0.17%     
===========================================
  Files          679      685       +6     
  Lines       108433   108844     +411     
===========================================
+ Hits         57378    57789     +411     
  Misses       51055    51055              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[TranscenderNing]

[TranscenderNing]

reftTrainer重写了compute_loss函数，前向传播过程中输入除了input_id， attention_maks, labels, 还需要unit_locations，表示干预哪些位置token的隐藏状态

[CLAassistant]

All committers have signed the CLA.

[lugimzzz]

ReFT和其他peft方法例如lora、vera的区别在于数据预处理略有不同、替换nn.linear变成添加hook，其实整体设计完全可以参考peft方法，让代码更加简洁直观

[lugimzzz]

单测挂了

[lugimzzz]

[TranscenderNing]

单测挂了