LoRA
LoRA copied to clipboard
The description and the behavior don't match
In the following code, the comment says weight A is init as usual (kaiming init as in other places in the code) and B as zeros. However, the behavior is quite the opposite:
https://github.com/microsoft/LoRA/blob/3f5c193f431c8a09448f0184f6f883ad393f22d0/loralib/layers.py#L58-L60
Is this intended? Thanks.