LyCORIS icon indicating copy to clipboard operation
LyCORIS copied to clipboard

Error Initializing t2 using `weight_gen` functinal API.

Open yaswanth19 opened this issue 1 year ago • 2 comments

We get an error "AttributeError: 'NoneType' object has no attribute 'shape'" when Initializing t2 parameter for Linear layer. This precisely happens because default value of t2 is None and for rank=1 or it is low when compared to shape then we are passing the None value for kaiming initialization.

Code to reproduce the Issue.

import torch
org_weight = torch.rand((10,20))
out = weight_gen(org_weight=org_weight,rank=1)

yaswanth19 avatar Oct 06 '24 19:10 yaswanth19

t2 will be "not None" if and only if:

  1. your input is not only 2 dim(such as for convolution)
  2. you provided tucker=True (default True)

KohakuBlueleaf avatar Oct 06 '24 22:10 KohakuBlueleaf

Not all the weight will be not None in all the case. it is expected.

KohakuBlueleaf avatar Oct 06 '24 22:10 KohakuBlueleaf