DeepLearningExamples icon indicating copy to clipboard operation
DeepLearningExamples copied to clipboard

Published 20 hours ago • verified publisher icon NVIDIA

init_weights has no effect after weight_norm

Open Andras7 opened this issue 11 months ago • 0 comments

Related to HifiGAN and more.

After some research I found out that initialization has no effect after weight_norm, I think it is possible to improve a lot on the bad default pytorch initialization. The order should be init_weights then weight_norm

Andras7 avatar Jul 21 '23 23:07 Andras7