LyCORIS
LyCORIS copied to clipboard
Error Initializing t2 using `weight_gen` functinal API.
We get an error "AttributeError: 'NoneType' object has no attribute 'shape'" when Initializing t2 parameter for Linear layer. This precisely happens because default value of t2 is None and for rank=1 or it is low when compared to shape then we are passing the None value for kaiming initialization.
Code to reproduce the Issue.
import torch
org_weight = torch.rand((10,20))
out = weight_gen(org_weight=org_weight,rank=1)
t2 will be "not None" if and only if:
- your input is not only 2 dim(such as for convolution)
- you provided tucker=True (default True)
Not all the weight will be not None in all the case. it is expected.