MLMAN icon indicating copy to clipboard operation
MLMAN copied to clipboard

请问能解释一下全连接层和lstm这样初始化的原因吗?

Open qq31415926 opened this issue 2 years ago • 0 comments

全连接层初始化代码 bias = np.sqrt(6.0 / (input_linear.weight.size(0) + input_linear.weight.size(1))) nn.init.uniform_(input_linear.weight, -bias, bias) if input_linear.bias is not None: input_linear.bias.data.zero_() lstm层初始化代码 for ind in range(0, input_lstm.num_layers): weight = eval('input_lstm.weight_ih_l' + str(ind)) bias = np.sqrt(6.0 / (weight.size(0) / 4 + weight.size(1))) nn.init.uniform_(weight, -bias, bias) weight = eval('input_lstm.weight_hh_l' + str(ind)) bias = np.sqrt(6.0 / (weight.size(0) / 4 + weight.size(1))) nn.init.uniform_(weight, -bias, bias)

qq31415926 avatar Sep 28 '22 02:09 qq31415926