WeightStandardization icon indicating copy to clipboard operation
WeightStandardization copied to clipboard

It just explodes!!!

Open VCasecnikovs opened this issue 3 years ago • 3 comments

Hello, I've been testing WS on my dataset and on my network. I have read about std error. But even after using std = (torch.sqrt(torch.var(weight.view(weight.size(0), -1), dim=1) + 1e-12).view(-1, 1, 1, 1) + 1e-5). I found out that there is a problem with input exploding. When I use basic Conv2D block this problem does not exist. So, my question is. Is it possible to somehow to figure it out?

VCasecnikovs avatar Jul 23 '20 14:07 VCasecnikovs