peft icon indicating copy to clipboard operation
peft copied to clipboard

A bug caused by `endswith` when using modules_to_save.

Open gary-young opened this issue 7 months ago • 4 comments

System Info

I find in the line 326 in the file peft/src/peft/utils/other.py, where the modules_to_save modules should be set trainable and here a endswith is used to determine whether a modules has been listed in the modules to save. However, I find a very special case that if there are two modules with the same end string but only one of them is listed in the modules_to_save. The endswith cannot correct distinct them. For example, in my code, there are two modules with name "model.layers.0.xxxx" and another with name "img_model.layers.0.xxxx", and it cause a bug that the latter was also mistakenly set to trainable.

I don't know why the endswith will be used here. But I think it is common to coincidentally have two modules with the same ending.

Who can help?

@BenjaminBossan

Information

  • [ ] The official example scripts
  • [X] My own modified scripts

Tasks

  • [ ] An officially supported task in the examples folder
  • [X] My own task or dataset (give details below)

Reproduction

A model with two different modules with the same ending string. Only one of them is in the modules_to_save.

Expected behavior

Only the one in the modules_to_save should be set as trainable.

gary-young avatar Jul 09 '24 21:07 gary-young