SFace icon indicating copy to clipboard operation
SFace copied to clipboard

Initialization method of weights

Open CloudWalking0 opened this issue 3 years ago • 0 comments

Hi, yy, I was wondering why you choose xavier_normal_(self.weight, gain=2, mode='out') instead of nn.init.xavier_uniform_(self.weight) when initializing weights. And by looking through the xavier_normal_ function, I found the weights won't participate in gradient propagation, why?

Thanks for your sharing, and have a nice day :)

CloudWalking0 avatar Sep 01 '21 07:09 CloudWalking0