EasyEdit icon indicating copy to clipboard operation
EasyEdit copied to clipboard

关于wise.py

Open zhengguanyu opened this issue 6 months ago • 3 comments

非常感谢工作的分享,很有启发!有两个问题想请教:

  1. 观察到论文和代码中只复制了一个层的weight,请问有考虑过优化两个层的weight吗?比如 transformer.h[7].mlp.c_fc.weighttransformer.h[8].mlp.c_fc.weight。如果想修改的话,主要在wise.py中修改吗?
  2. 请问有考虑过引入bias吗?联合更新weight和bias

zhengguanyu avatar Aug 10 '24 09:08 zhengguanyu