LSUV-keras
LSUV-keras copied to clipboard
Initialization failure
D:\jupyter notebook\lsuv_init.py in LSUVinit(model, batch, verbose, margin, max_iter) 64 weights_and_biases = layer.get_weights() 65 weights_and_biases[0] /= np.sqrt(variance) / np.sqrt(needed_variance) ---> 66 layer.set_weights(weights_and_biases) 67 weights /= np.sqrt(variance) / np.sqrt(needed_variance) 68 layer.set_weights([weights, biases])
UnboundLocalError: local variable 'weights' referenced before assignment
It should be changed to following :
layer.set_weights(weights_and_biases) weights_and_biases[1] /= np.sqrt(variance) / np.sqrt(needed_variance) layer.set_weights(weights_and_biases)
but [1] actually has biases of the layer, and according to the paper, we're supposed to just scale the weights not the biases, so why is the code initializing biases as well ?
Just remove lines 66 and 67 to make it work again:
weights /= np.sqrt(variance) / np.sqrt(needed_variance)
layer.set_weights([weights, biases])