WeightStandardization
WeightStandardization copied to clipboard
It just explodes!!!
Hello, I've been testing WS on my dataset and on my network. I have read about std error. But even after using std = (torch.sqrt(torch.var(weight.view(weight.size(0), -1), dim=1) + 1e-12).view(-1, 1, 1, 1) + 1e-5). I found out that there is a problem with input exploding. When I use basic Conv2D block this problem does not exist. So, my question is. Is it possible to somehow to figure it out?