LSUV-keras icon indicating copy to clipboard operation
LSUV-keras copied to clipboard

Initialization failure

Open dalinzhangzdl opened this issue 5 years ago • 2 comments

D:\jupyter notebook\lsuv_init.py in LSUVinit(model, batch, verbose, margin, max_iter) 64 weights_and_biases = layer.get_weights() 65 weights_and_biases[0] /= np.sqrt(variance) / np.sqrt(needed_variance) ---> 66 layer.set_weights(weights_and_biases) 67 weights /= np.sqrt(variance) / np.sqrt(needed_variance) 68 layer.set_weights([weights, biases])

UnboundLocalError: local variable 'weights' referenced before assignment

dalinzhangzdl avatar Jul 07 '19 02:07 dalinzhangzdl

It should be changed to following :

layer.set_weights(weights_and_biases) weights_and_biases[1] /= np.sqrt(variance) / np.sqrt(needed_variance) layer.set_weights(weights_and_biases)

but [1] actually has biases of the layer, and according to the paper, we're supposed to just scale the weights not the biases, so why is the code initializing biases as well ?

shaginhekvs avatar Jul 11 '19 08:07 shaginhekvs

Just remove lines 66 and 67 to make it work again:

            weights /= np.sqrt(variance) / np.sqrt(needed_variance)
            layer.set_weights([weights, biases])

andrasmolnar avatar Jul 23 '19 12:07 andrasmolnar