Can not reproduce the results of PARAMETER UPDATE WITH SEQUENTIAL LOW-RANK APPROXIMATION

[gaoxiangxiang1996]

Thank you for your outstanding work. However, we are unable to reproduce the results of PARAMETER UPDATE WITH SEQUENTIAL LOW-RANK APPROXIMATION based on the code you published. Our steps are as follows: First, (1) set the target modules to q_u_proj, k_u_proj, v_u_proj, o_u_proj, gate_u_proj, down_u_proj, and up_u_proj to update U. Then, (2) merge the model with the updated U into the main model. After that, (3) set the target modules in the merged model to q_v_proj, k_v_proj, v_v_proj, o_v_proj, gate_v_proj, down_v_proj, and up_v_proj to perform the update. Finally, (4) merge the model with the updated V into the main model again. Could you please let us know if there is an issue with any of these steps? We would greatly appreciate your guidance.