cruise icon indicating copy to clipboard operation
cruise copied to clipboard

Introduce various weight initialization methods

Open beomyeol opened this issue 8 years ago • 0 comments

In the current DNN implementation, the weights of layers such as fully connected layers and convolutional layers are initialized with random values from a Gaussian distribution. It is known that learning deep neural network is largely influenced by initial weight values as well as neural network architecture. Thus, we should introduce various methods to initialize weight values. For example, Caffe has "weight filler" which fills initial weight parameters of layers. There are many types of weight fillers such as Constant, Gaussian, Positive Unit ball, Uniform, Xavier, MSRA and Bilinear. We also introduce these methods to get better training performance of deep neural networks.

beomyeol avatar Jun 24 '16 05:06 beomyeol