LoRA icon indicating copy to clipboard operation
LoRA copied to clipboard

The description and the behavior don't match

Open yongchanghao opened this issue 8 months ago • 2 comments

In the following code, the comment says weight A is init as usual (kaiming init as in other places in the code) and B as zeros. However, the behavior is quite the opposite:

https://github.com/microsoft/LoRA/blob/3f5c193f431c8a09448f0184f6f883ad393f22d0/loralib/layers.py#L58-L60

Is this intended? Thanks.

yongchanghao avatar Oct 26 '23 05:10 yongchanghao